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LIST OF ACRONYMS 


Aetna Better Aetna Better Health of Missouri 


Health 

BHO Behavioral Health Management Organization 

CAHPS Consumer Assessment of Health Plans Survey 

CDC Centers for Disease Control and Prevention 

CHI-SQUARE psasiek ecraaen nel val Gees the probability of a change or 

Cl Confidence Interval 

CMHC Community Mental Health Center 

CMS Centers for Medicare and Medicaid Services, U.S. Department of 
Health and Human Services 

CY Calendar Year 

DHHS U.S. Department of Health and Human Services 

DHSS Missouri Department of Health and Senior Services 

DSS Missouri Department of Social Services 

EQR External Quality Review 

EQRO External Quality Review Organization 

FFS MO HealthNet Fee-for-Service 

HEDIS Healthcare Effectiveness Data and Information Set 

HIPAA Health Insurance Portability and Accountability Act 

HIS Health Information Systems 

HMO Health Maintenance Organization 

pee Home State Health Plan of Missouri 

ISCA Information Systems Capability Assessment 

LPHA Local Public Health Agency 
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MC+ 


MC+ MCOs 


MCHP 


MCO 


MDIFP 


MO HEALTHNET 


MO HEALTHNET 
MCHPs 
MISSOURI CARE 
MOHSAIC 

NCQA 


N.S. 


PCP 


PIHP 


PIP 


QA & | 


QI/UM 
Coordinator 


SMA 


SPHA 


The name of the Missouri Medicaid Program for families, children, and 
pregnant women, prior to July 2007. 


Missouri Medicaid Program Managed Care Organizations (prior to July 
2007) 


Managed Care Health Plan 
Managed Care Organization 


Missouri Department of Insurance, Financial Institutions and 
Professional Registration 


The name of the Missouri Medicaid Program for families, children, and 
pregnant women. 


Missouri Medicaid Program Managed Care Health Plans 


Missouri Care Health Plan 


Missouri Public Health Integrated Information System 


National Committee for Quality Assurance 


Not significant, indicating that a statistical test does not result in the 
ability to conclude that a real effect exists. 


Primary Care Provider 


Prepaid Inpatient Health Plan 


Performance Improvement Project 


MO HealthNet Managed Care Quality Assessment and Improvement 
Advisory Group 


Quality Improvement/Utilization Management Coordinator 


State Medicaid Agency, the Missouri Department of Social Services, MO 
HealthNet Division 


State Public Health Agency, the Missouri Department of Health and 
Senior Services 
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Administrative Method 


Confidence interval 


Hybrid Method 


Interrater reliability 


(IRR) 


Probability sample 


Random sample 


Reliability 


Sampling frame 


Sampling unit 


Simple sample 


The Administrative Method of calculating HEDIS Performance Measures 
requires the MCHP to identify the denominator and numerator using 
transaction data or other administrative databases. The Administrative 
Method outlines the collection and calculation of a measure using only 
administrative data, including a description of the denominator (i.e., the 
entire eligible population), the numerator requirements (i.e., the 
indicated treatment or procedure) and any exclusion(s) allowed for the 


measure. 
The range of accuracy of a population estimate obtained from a sample. 


Hybrid Method requires the MCHP to identify the numerator through 
both administrative and medical record data. The MCHP reports a rate 
based on members in the sample who are found through either 
administrative or medical record data to have received the service 


identified in the numerator. 


A method of addressing the internal validity of a study by ensuring that 
data are collected in a consistent manner across data collectors. 

A sample in which every element in the sampling frame has a known, 
non-zero probability of being included in a sample. This produces 
unbiased estimates of population parameters that are linear functions of 
the observations from the sample data!. 

Selection of sampling units from a sampling frame where each unit has 
an equal probability of selection. 

The consistency of findings across time, situations, or raters. 

The population of potential sampling units that meet the criteria for 
selection (e.g., Medical encounter claim types from January |, 2004 
through March 31, 2004). 


Each unit in the sampling frame (e.g., an encounter). 


Selection of sampling units from one sampling frame. 





1Levy, P.S., Lemeshow, S. (1999). Sampling of Populations: Methods and Applications, Third Edition. John 


Wiley and Sons: New York. 
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PREPARATION WITH THE STATE MEDICAID AGENCY 

Effective January |, 2016 the State of Missouri contract for the External Quality Review of the MO 
HealthNet Managed Care Program (State of Missouri Contract No: C312155001, Amendment No.: 
004) was awarded to comply with federal requirements for states to contract with an external, 
independent entity to implement the mandatory protocols for External Quality Review. Meetings 
for planning the scope of work, technical methods and objectives, are scheduled beginning each 
January for the upcoming review year. Meetings are held with the SMA and the EQRO throughout 
the review period. Additional meetings and teleconference calls may be conducted as needed 


between MO HealthNet, the State Medicaid Agency (SMA) and EQRO personnel. 


At the first meeting of each year, the previous years’ report is discussed and the plan for the 
subsequent audit is initiated. The EQRO clarifies the SMA’s objectives for each of the protocols, 
develops data requests, prepares detailed proposals for the implementation and analysis of data for 
each protocol, and prepares materials for SMA review. Plans are made to conduct Orientation 
Conference Calls for the upcoming EQR with each MO HealthNet Managed Care Health Plan 
(MCHP) that are attended by the SMA. Written proposals for each protocol are developed and 


approved by the SMA indicating differences in the approach or information to be validated. 


PREPARATION OF MCHPS 

To prepare the MCHPs for the implementation of the yearly EQR an annual Orientation 
Conference Call is conducted by the EQRO Project Director and personnel. The EQRO Project 
Director and personnel conduct orientation to the protocols and the EQR processes with each 
MCHP. In addition, the EQRO Project Director presents a timeline for project implementation and 
answers MCHP questions at a combined MO HealthNet Managed Care QA&l Advisory Group/MO 
HealthNet Managed Care All-Plan meeting. 


The EQRO Assistant Project Director arranges the dates of the teleconference calls with MCHP 
QI/UM Coordinators or Plan Administrators. A detailed presentation, tentative list of data 
requests, and the proposals approved by the MO HealthNet are sent to MCHPs prior to the 
teleconference orientation sessions. MCHPs are requested to have all personnel involved in fulfilling 
the requests or in implementing activities related to the protocols (e.g., performance improvement 
projects to be validated, performance measures to be validated) present at the teleconference calls. 


The orientation presentation is contained in Appendix |. A MO HealthNet representative is invited 
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to attend all conference calls. Notes are sent regarding any calls that the representative does not 
attend. To avoid confusion and the inundation of multiple requests at once, the requests for 
information from MCHPs are normally implemented in a staged approach from January through 
April. All communications (letters, general and specific instructions) are approved by MO 


HealthNet prior to sending them to the MCHPs. 


DEVELOPMENT OF WORKSHEETS, TOOLS, AND RATING CRITERIA 

The EQRO Project Director, Assistant Project Director, and a healthcare consultant are responsible 
for modifying the worksheets and tools used by the EQRO during each audit. The EQRO Assistant 
Project Director revises the worksheet (Attachment B) for Validating Performance Improvement 


Project Protocol to add details specific to the MO HealthNet Managed Care Program each year. 


The Validating Performance Measures Protocol worksheets are revised and updated by the EQRO 
Project Director to reflect the Performance Measures selected for review for the appropriate 
HEDIS year. The worksheets developed by Behavioral Health Concepts Inc. staff are updated 


annually to reflect the information needed for that year’s audit. 


MO HealthNet continues to conduct the activities of the MO HealthNet Managed Care Compliance 
with Managed Care Regulations Protocol through the state contract compliance monitoring process. 
The work of the EQRO involves the review and evaluation of this information (see Medicaid 
Program; External Quality Review of Medicaid Managed Care Organizations of 2003, CFR §438.58). 
The state contract for the EQRO requires the review of MO HealthNet’s activities with regard to 
the Protocol. Additional policies and documents are requested prior to and during the on-site visits 
with MCHPs when information was incomplete or unclear. To facilitate the review of compliance 
with federal regulations, the EQRO Assistant Project Director works with MO HealthNet staff to 
develop the focus of each year’s compliance review to ensure that it addresses issues of concern 
where compliance may be compromised. Focused interview tools are developed and submitted to 
MO HealthNet for review and approval. The MO HealthNet Managed Care Program consultant, 
who participates as part of the EQRO team each year reviews and assists in refinement of 


compliance activities. 


The EQRO utilizes the rating system developed during the 2004 audit to provide ratings for each 


MCHP’s compliance. MO HealthNet provides information on MCHP policy compliance with state 
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contract requirements annually. The EQRO determines if this meets the policy requirements of the 
federal regulations. The EQRO staff and the consultant review all available materials and meet with 
MO HealthNet staff to clarify any of their comments and compliance ratings. Issues are identified 
for follow-up at site visits. Updates on MCHP compliance are accepted up until the time of the on- 
site reviews to ensure that the EQRO has up-to-date information. Recommended ratings, based 


upon the preapproved rating scale are provided to SMA. 


REVIEWERS 

Three reviewers are utilized to complete all sections of the EQR. Interviews, document review, and 
data analysis activities for the Validating Performance Measure Protocol were performed by two 
reviewers from the External Quality Review Organization (EQRO). The Project Director 
conducted interviews, document review, and data analysis; she is a licensed attorney with a graduate 
degree in Health Care Administration, as well as seventeen years’ experience in public health and 


managed care in four states. This is her twelfth External Quality Review. 


Two reviewers take primary responsibility for conducting the Performance Improvement Project 
(PIP) Validation and the Compliance Protocol activities, including interviews and document review. 
The External Quality Review Organization (EQRO) Project Director conducts backup activities, 
including assistance during the interview process, and oversight of the PIP and Compliance Protocol 
team. All reviewers are familiar with the federal regulations and the manner in which these were 


operationalized by the MO HealthNet Managed Care Program to comply with the federal protocols. 


The following sections summarize the aggregate findings and conclusions for each of the mandatory 
protocols. The full report is organized according to each protocol and contains detailed 
descriptions of the findings and conclusions (defined as the Quality Care, Access to Care, Timeliness 
of Care, and recommendations). In addition, it provides MCHP to MCHP comparisons and MCHP 


summaries for each protocol. 
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PURPOSE AND OBJECTIVES 


The purpose of the Performance Improvement Project (PIP) is to assess and improve the processes 
and outcomes of health care provided by the health plan. The review is developed to determine 
whether the health care quality PIP was designed, conducted, and reported in a methodologically 
sound manner. The EQRO uses the procedures outlined in the CMS EQR Protocol 3 regarding 


PIPs. The EQRO assesses the validity and reliability of the results presented by the MCHPs. 


Each MCHP is required to conduct, at a minimum, at least one clinical and one non-clinical PIP 
during each calendar year. MCHPs may engage in multiple projects over multiple years. However, if 
an on-going PIP is reviewed, at least one new activity is required to enhance ongoing quality 


improvement, and the PIP documentation must be updated accordingly. 


PIP topics and methodologies are to reflect relevant clinical, administrative and population-based 


improvement efforts to improve health care delivery and outcomes for the people served. 


TECHNICAL METHODS 


There are three evaluation activities specified in the protocol for Validating 
Performance Improvement Projects. 

“Activity One: Assessing the MCOs/PIHPs Methodology for Conducting the 
PIP” consists of ten steps: 





Step One: Review the selected study topic(s) 

Step Two: Review the study question(s) 

Step Three: Review selected study indicator(s) 

Step Four: Review the identified study population 

Step Five: Review sampling methods (if sampling was used) 

Step Six: Review the data collection procedures 

Step Seven: Assess the MCOs improvement strategies 

Step Eight: Review the data analysis and interpretation of study results 
Step Nine: Assess the likelihood that reported improvement is “real” 
improvement 

10. Step Ten: Assess the sustainability of documented improvement 


I. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 





“Activity Two: Verifying PIP Study Findings” is optional, and involves auditing PIP data. “Activity 
Three: Evaluate Overall Reliability and Validity of Study Findings” involves accessing whether the 
results and conclusions drawn from the PIPs are valid and reliable. Activities One and Three are 


conducted by the EQRO. 
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TIME FRAME AND SELECTION 

Two projects that were underway during the preceding |2 months at each MCHP are selected for 
validation. One project is to be clinical in nature, and one non-clinical. The projects to be validated 
are reviewed with MO HealthNet and EQRO staff after topic submission is complete. The intent is 
to identify projects which are mature enough for validation (i.e., planned and in the initial stages of 
implementation), underway or completed during the previous calendar year. MO HealthNet makes 
the final decision regarding the actual PIPs to be validated from the descriptions submitted by the 


MCHPs. The non-clinical PIP currently reviewed for each MCHP is their approach to a Statewide 
PIP. 


PROCEDURES FOR DATA COLLECTION 
The evaluation involves review of all materials submitted by the MCHPs including, but not limited to, 
the materials listed below. During the training teleconferences MCHPs are encouraged to review 


Attachment A of the Validating Performance Improvement Projects Protocol, to ensure that they 





include supporting documents, tools, and other information necessary to evaluate the projects 
submitted, based on this tool. 

= Narrative descriptions 

* Problem identification 

= Hypotheses 

= Study questions 

" Description of interventions(s) 

= Methods of sampling 

"Study design planned analysis 

* Planned interventions 

= Sample tools, measures, survey, etc. 

"Baseline data source and data 

"Cover letter with clarifying information 

"Overall analysis of the validity and reliability of each study 


=" Evaluation of the results of the PIPs 


The EQRO Project Director, Assistant Project Director, and Nurse Consultant meet with the 
MCHBP staff responsible for planning, conducting, and interpreting the findings of the PIPs during the 


on-site reviews occurring annually. The review focuses on the findings of projects conducted. 
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MCHPs are instructed that additional information and data, not available at the time of the original 
submission, can be provided at the on-site review or shortly thereafter. The time scheduled during 
the on-site review is utilized to conduct follow-up questions, to review data obtained, and to 
provide technical assistance to MCHPs regarding the planning, implementation and credibility of 
findings from PIPs. In addition, individual clarifying questions are used to gather more information 
regarding the PIPs during the on-site interviews. The following questions were formulated and 
answered in the original documentation, or are posed to the MCHPs during the on-site review: 
e How were the interventions determined and why did the MCHP choose this approach? 
e What are the conclusions about the effectiveness of the interventions to date? 
e How were the outcomes interpreted and linked to the interventions? 
e Discuss the effects of these interventions and how they impacted services to members. 
e How are the PIP interventions and goals communicated throughout MCHP staff? Are all 
staff, including case managers and customer services personnel, involved? 
e What instruments are used for data collection? 
e How were accuracy, consistency, and validity assured? 
e What did the MCHP hope to learn from the findings relevant to the MO HealthNet 
Managed Care population? 


e How was improvement analyzed? 


All PIPs are evaluated by the Assistant Project Director, in consultations with the Project Director. 
In addition, the projects are reviewed with follow-up suggestions posed by the Project Director, 


who approves final ratings based on all information available to the team. 


ANALYSIS 
Criteria for identification of a PIP as outlined in the CMS protocols include the following: 
e PIPs need to have a pre-test, intervention, and post-test. 
e PIPs need to control for extraneous factors. 
e PIPs need to include an entire population. 
e Pilot projects do not constitute a PIP. 
e Satisfaction studies alone do not constitute a PIP. 
e Focused studies are not PIPs: A focused study is designed to assess processes and 
outcomes on one-time basis, while the goal of a PIP is to improve processes and outcomes 


of care over time. 
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The Managed Care contract describes the following requirements for MCHP’s relative to 

conducting PIPs: 
Performance Improvement Projects: The MCHP shall conduct performance 
improvement projects that are designed to achieve, through ongoing measurements 
and intervention, significant improvement, sustained over time, in clinical care and 
nonclinical care areas that are expected to have a favorable effect on health 
outcomes and member satisfaction. As requested, the MCHP shall report the status 
and results of each performance improvement project to the state agency, which 
must include state and/or MCHP designated performance improvement projects... 


The performance improvement projects must involve the following: 


e Measurement of performance using objective quality indicators. 

e Implementation of system interventions to achieve improvement in quality. 

e Evaluation of the effectiveness of the interventions. 

e Planning and initiation of activities for increasing or sustaining improvement. 

e Completion of the performance improvement project in a reasonable time 
period so as to generally allow information on the success of performance 
improvement projects in the aggregate to produce new information on quality 
of care every year. 

e Performance measures and topics for performance improvement projects 


specified by CMS in consultation with the state agency and other stakeholders. 


All PIPs submitted by MCHPs prior to the site visits are reviewed using an expanded version of the 
checklist for conducting Activity One, Steps | through 10, and Activity Three (Judgment of the 


Validity and Reliability of the PIPs) of the Validating Performance Improvement Projects Protocol, 





Attachment A. Because certain criteria may not be applicable for projects that are underway at the 
time of the review, some specific items may be considered as “Not Applicable.” Criteria are rated 
as “Met” if the item was applicable to the PIP, if documentation is available that addresses the item, 
and if the item could be deemed Met based on the study design. The proportion of items rated as 
“Met” is compared to the total number of items applicable for the particular PIP. Given that some 
PIPS may be underway in the first year of implementation, it is not possible to judge or interpret 
results; validity of improvement; or sustained improvements (Steps 8-10) in all instances. The final 


evaluation of the validity and reliability of studies is based on the potential for the studies to produce 
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credible findings. Detailed recommendations and suggestions for improvement are made for each 


item where appropriate, and are presented in the individual MCHP summaries. Some items are 


rated as “Met” but continue to include suggestions and recommendations as a method of improving 


the information presented. The following are the general definitions of the ratings developed for 


evaluating the PIPs. 


Met: 


Partially Met : 


Not Met: 


Not Applicable: 


Credible, reliable, and valid methods for the item were documented. 


Credible, reliable, or valid methods were implied or able to be established 
for part of the item. 


The study did not provide enough documentation to determine whether 
credible, reliable, and valid methods were employed; errors in logic were 


noted; or contradictory information was presented or interpreted 
erroneously. 


Only to be used in Step 5, when there is clear indication that the entire 

population was included in the study and no sampling was conducted; or 

in Steps 8 through 10 when the study period was underway for the first 
ear. 





2| 
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TECHNICAL METHODS 

Reliable and valid calculation of performance measures is a critical component to the EQRO audit. These 
calculations are necessary to calculate statewide rates, compare the performance of MCHPs with other 
MCHPs, and to compare State and MCHP performance with national benchmarked data for Medicaid 
Managed Care and/or Commercial Managed Care Organization members. These types of comparisons 
allow for better evaluation of program effectiveness and access to care. The EQRO reviews the selected 
data to assess adherence to State of Missouri requirements for MCHP performance measurement and 
reporting. The Missouri Code of State Regulations (19 CSR §10-5.010 Monitoring Health Maintenance 
Organizations) contains provisions requiring all Health Maintenance Organizations (HMOs) operating in the 
State of Missouri to submit to the SPHA member satisfaction survey findings and quality indicator data in 
formats conforming to the National Committee for Quality Assurance (NCQA) Health Employer Data 
Information Set (HEDIS) Data Submission Tool (DST) and all other HEDIS Technical Specifications> for 
performance measure descriptions and calculations. The State of Missouri contract for MO HealthNet 
Managed Care (C306122001, Revised Attachment 6, Quality Improvement Strategy) further stipulates that 
MO HealthNet MCHPs will follow the instructions of the SPHA for submission of HEDIS measures. Three 
performance measures are selected by MO HealthNet for validation annually. These measures are required 
to be calculated and reported by MCHPs to MO HealthNet. HEDIS based measures are also required to be 
reported to the SPHA for MO HealthNet Managed Care Members. A review is conducted for each of the 
three measures selected based upon the Technical Specifications. These specifications are provided in the 


following tables: 
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HEDIS 2016 PRENATAL AND POSTPARTUM CARE (PPC) 

Description 

The percentage of deliveries of live births between November 6 of the year prior to the measurement year 
and November 5 of the measurement year. For these women, the measure assesses the following facets of 
prenatal and postpartum care. 

Timeliness of Prenatal Care. The percentage of deliveries that received a prenatal care visit as a member of the 
organization in the first trimester or within 42 days of enrollment in the organization. 

Postpartum Care. The percentage of deliveries that had a postpartum visit on or between 21 and 56 days 
after delivery. 


Eligible Population 


Product lines Commercial, Medicaid (report each product line separately). 
Age None specified. 

Continuous 43 days prior to delivery through 56 days after delivery. 
enrollment 

Allowable gap No allowable gap during the continuous enrollment period. 
Anchor date Date of delivery. 

Benefit Medical. 


Event/ diagnosis Delivered a live birth on or between November 6 of the year prior to the measurement year 
and November 5 of the measurement year. Include women who delivered in any setting. 
Multiple births. Women who had two separate deliveries (different dates of service) 
between November 6 of the year prior to the measurement year and November 5 of 
the measurement year count twice. Women who had multiple live births during one 
pregnancy count once. 

Follow the steps below to identify the eligible population, which is the denominator for 
both rates. 

Step | Identify deliveries. Identify all women with a delivery (Deliveries Value Set) between 
November 6 of the year prior to the measurement year and November 5 of the 
measurement year. 

Step 2 Exclude non-live births (Non-live Births Value Set). 

Step 3 Identify continuous enrollment. Determine if enrollment was continuous between 43 
days prior to delivery and 56 days after delivery, with no gaps. 








Administrative Specification 

Denominator The eligible population. 

Numerator 

Timeliness of — A prenatal visit in the first trimester or within 42 days of enrollment, depending on the 

Prenatal Care date of enrollment in the organization and the gaps in enrollment during the pregnancy. 
Include only visits that occur while the member was enrolled. 
Follow the steps below to identify the numerator. 

Step | Determine enrollment status during the first trimester. For all women in the eligible 
population, identify those who were enrolled on or before 280 days prior to delivery (or 
estimated date of delivery [EDD]). For these women, proceed to step 2. 
For women not enrolled on or before 280 days prior to delivery (or EDD), who were 
therefore pregnant at the time of enrollment, proceed to step 3. 
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Step 2 


Step 3 


Step 4 


Step 5 


Determine continuous enrollment for the first trimester. Identify women from step | who 
were continuously enrolled during the first trimester (176-280 days prior to delivery [or 
EDD)), with no gaps in enrollment. For these women, determine numerator compliance 
using the decision rules for Identifying Prenatal Care For Women Continuously Enrolled During 
the First Trimester. 

For women who were not continuously enrolled during the first trimester (e.g., had a gap 
between 176 and 280 days before delivery), proceed to step 3. 

Determine the start date of the last enrollment segment (i.e., the enrollment segment 
during the pregnancy with the start date that is closest to the delivery date). 

For women whose last enrollment started on or between 219 and 279 days before delivery, 
proceed to step 4. 

For women whose last enrollment started less than 219 days before delivery, proceed to step 5. 
Determine numerator compliance. If the last enrollment segment started on or between 
219 and 279 days before delivery, determine numerator compliance using the instructions 
for Identifying Prenatal Care for Women Not Continuously Enrolled During the First Trimester 
and find a visit between the last enrollment start date and 176 days before delivery. 
Determine numerator compliance. If the last enrollment segment started less than 219 
days before delivery (i.e., between 219 days before delivery and the day of delivery), 
determine numerator compliance using the instructions for Identifying Prenatal Care for 
Women Not Continuously Enrolled During the First Trimester and find a visit within 42 
days after enrollment. 


Identifying Prenatal Care for Women Continuously Enrolled During the First Trimester 


Decision Rule 
I 


Decision Rule 
2 


Decision Rule 
3 


Either of the following during the first trimester, where the practitioner type is an 
OB/GYN or other prenatal care practitioner or PCP meets criteria: 

A bundled service (Prenatal Bundled Services Value Set) where the organization can 
identify the date when prenatal care was initiated (because bundled service codes are 
used on the date of delivery, these codes may be used only if the claim form indicates 
when prenatal care was initiated). 

A visit for prenatal care (Stand Alone Prenatal Visits Value Set). 

Any of the following during the first trimester, where the practitioner type for the 
prenatal visit is an OB/GYN or other prenatal care practitioner, meet criteria: 

A prenatal visit (Prenatal Visits Value Set) with an obstetric panel (Obstetric Panel Value 
Set). 

A prenatal visit (Prenatal Visits Value Set) with an ultrasound (echocardiography) of the 
pregnant uterus (Prenatal Ultrasound Value Set). 

A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set). 

A prenatal visit (Prenatal Visits Value Set) with all of the following: 

Toxoplasma (Toxoplasma Antibody Value Set). 

Rubella (Rubella Antibody Value Set). 

Cytomegalovirus (Cytomegalovirus Antibody Value Set). 

Herpes simplex (Herpes Simplex Antibody Value Set). 

A prenatal visit (Prenatal Visits Value Set) with rubella (Rubella Antibody Value Set) and 
ABO (ABO Value Set). 

A prenatal visit (Prenatal Visits Value Set) with rubella (Rubella Antibody Value Set) and 
Rh (Rh Value Set). 

A prenatal visit (Prenatal Visits Value Set) with rubella (Rubella Antibody Value Set) and 
ABO/Rh (ABO and Rh Value Set). 

Any of the following during the first trimester, where the practitioner type is a PCP, 
meet criteria: 
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A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and an obstetric panel (Obstetric Panel Value Set). 
A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and an ultrasound (echocardiography) of the pregnant 
uterus (Prenatal Ultrasound Value Set). 

A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and all of the following: 

Toxoplasma (Toxoplasma Antibody Value Set). 

Rubella (Rubella Antibody Value Set). 

Cytomegalovirus (Cytomegalovirus Antibody Value Set). 

Herpes simplex (Herpes Simplex Antibody Value Set). 



































A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and rubella (Rubella Antibody Value Set) and ABO 
(ABO Value Set). 

A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and rubella (Rubella Antibody Value Set) and Rh (Rh 
Value Set). 

A prenatal visit (Prenatal Visits Value Set) with a pregnancy-related diagnosis code 
(Pregnancy Diagnosis Value Set) and rubella (Rubella Antibody Value Set) and ABO/Rh 
(ABO and Rh Value Set). 

A prenatal visit (Prenatal Visits Value Set) with any internal organization code for LMP 
or EDD with an obstetrical history. 

A prenatal visit (Prenatal Visits Value Set) with any internal organization code for LMP 
or EDD with risk assessment and counseling/education. 

Note: For Decision Rule 3 criteria that require a prenatal visit code (Prenatal Visits Value Set) 
and a pregnang-related diagnosis code (Pregnancy Diagnosis Value Set), codes must be on 
the same claim. 












































Identifying Prenatal Care for Women Not Continuously Enrolled During the First Trimester 
Any of the following, where the practitioner type is an OB/GYN or other prenatal care practitioner or PCP, 
meet criteria: 

A bundled service (Prenatal Bundled Services Value Set) where the organization can identify the date when 
prenatal care was initiated (because bundled service codes are used on the date of delivery, these codes may 
be used only if the claim form indicates when prenatal care was initiated). 

A visit for prenatal care (Stand Alone Prenatal Visits Value Set). 

A prenatal visit (Prenatal Visits Value Set) with an ultrasound (echocardiography) of the pregnant uterus 
(Prenatal Ultrasound Value Set). 

A prenatal visit (Prenatal Visits Value Set) with a principal pregnancy-related diagnosis code (Pregnancy 
Diagnosis Value Set). 

Note: For criteria that require a prenatal visit code (Prenatal Visits Value Set) and a pregnancy-related diagnosis 
code (Pregnancy Diagnosis Value Set), codes must be on the same claim. 
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YES 





STEP 3 
Was the member continuously 
enrolled for 176 to 280 days 
prior to delivery, with no gaps 
during this period? 











YES 





STEP 4 
Use the three decision rules for 
Identifying Prenatal Care for Women 
Continuously Enrolled During the 
First Trimester to determine 
numerator compliance. 








Postpartum Care 


Performance Measures 








STEP 1 
Identify the eligible 
population. 














STEP 2 


Was the member identified in step 1 enrolled on or 
before 280 days prior to delivery (or EDD)? 








NO 


NO 





STEP 5 
Determine the last 
enrollment segment. 

















Was the last enrollment 
segment less than 219 
days prior to delivery? 


YES 








NO 








Did the last enrollment 
segment begin on our 
between 219 and 279 days 
prior to delivery? 


YES 








A postpartum visit (Postpartum Visits Value Set). 
Cervical cytology (Cervical Cytology Value Set). 
A bundled service (Postpartum Bundled Services Value Set) where the 
organization can identify the date when postpartum care was rendered (because 
bundled service codes are used on the date of delivery, not on the date of the 
postpartum visit, these codes may be used only if the claim form indicates when 
postpartum care was rendered). 
Note: The practitioner requirement only applies to the Hybrid Specification. The 
organization is not required to identify practitioner type in administrative data. 











STEP 6 
Use Prenatal Care for Women Not 
Continuously Enrolled During the 
First Trimester and find a visit 
within 42 days after enrollment. 














STEP 7 
Use Prenatal Care for Women Not 
Continuously Enrolled During the 
First Trimester and find a visit 
between the last enrollment 
start date and 176 days 
before delivery. 











e A postpartum visit for a pelvic exam or postpartum care on or between 21 
and 56 days after delivery. Any of the following meet criteria: 
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Hybrid Specification 


Denominator 


Numerator 
Timeliness of 
Prenatal Care 


Administrative 





Medical record 





Postpartum Care 


Administrative 





A systematic sample drawn from the eligible population for each product line. 
Organizations may reduce the sample size using the current year’s lowest product 
line-specific administrative rate of these two indicators and the >81% indicator from 
Frequency of Ongoing Prenatal Care or the prior year’s lowest audited product line- 
specific rate for these two indicators and the >81% indicator from Frequency of 
Ongoing Prenatal Care. 


A prenatal visit in the first trimester or within 42 days of enrollment, depending on 
the date of enrollment in the organization and gaps in enrollment during the 
pregnancy. Include only visits that occurred while the member was enrolled. 

Refer to Administrative Specification to identify positive numerator hits from the 
administrative data. 

Prenatal care visit to an OB/GYN or other prenatal care practitioner or PCP. For 
visits to a PCP, a diagnosis of pregnancy must be present. Documentation in the 
medical record must include a note indicating the date when the prenatal care visit 
occurred, and evidence of one of the following. 

A basic physical obstetrical examination that includes auscultation for fetal heart 
tone, or pelvic exam with obstetric observations, or measurement of fundus height 
(a standardized prenatal flow sheet may be used). 

Evidence that a prenatal care procedure was performed, such as: 

Screening test in the form of an obstetric panel (must include all of the following: 
hematocrit, differential WWBC count, platelet count, hepatitis B surface antigen, 
rubella antibody, syphilis test, RBC antibody screen, Rh and ABO blood typing), or 
TORCH antibody panel alone, or 

A rubella antibody test/titer with an Rh incompatibility (ABO/Rh) blood typing, or 
Echography of a pregnant uterus. 

Documentation of LMP or EDD in conjunction with either of the following. 
Prenatal risk assessment and counseling/education. 

Complete obstetrical history. 

Note: For women whose last enrollment segment was after 219 days prior to delivery (i.e., 
between 219 days prior to delivery and the day of delivery) and women who had a gap 
during the first trimester, count documentation of a visit to an OB/GYN, family practitioner 
or other PCP with a principal diagnosis of pregnancy. 

A postpartum visit for a pelvic exam or postpartum care on or between 21 and 56 
days after delivery, as documented through either administrative data or medical 
record review. 

Refer to Administrative Specification to identify positive numerator hits from the 
administrative data. 
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Medical record Postpartum visit to an OB/GYN practitioner or midwife, family practitioner or other 
PCP on or between 21 and 56 days after delivery. Documentation in the medical 
record must include a note indicating the date when a postpartum visit occurred and 
one of the following. 

Pelvic exam. 

Evaluation of weight, BP, breasts and abdomen. 

Notation of “breastfeeding” is acceptable for the “evaluation of breasts” component. 
Notation of postpartum care, including, but not limited to: 

Notation of “postpartum care,” “PP care,” “PP check,” “6-week check.” 

A preprinted “Postpartum Care” form in which information was documented during 
the visit. 





Note 

For women continuously enrolled during the first trimester (1 76—280 days before delivery with no gaps), the 
organization has sufficient opportunity to provide prenatal care in the first trimester. Any enrollment gaps in the 
second and third trimesters are incidental. 

Criteria for identifying prenatal care for women who were not continuously enrolled during the first trimester allow 
more flexibility than criteria for women who were continuously enrolled. 

For women whose last enrollment segment started on or between 219 and 279 days before delivery, the organization 
has sufficient opportunity to provide prenatal care by the end of the first trimester. 

For women whose last enrollment segment started less than 219 days before delivery, the organization has sufficient 
opportunity to provide prenatal care within 42 days after enrollment. 

Services that occur over multiple visits count toward this measure if all services are within the time frame established 
in the measure. Ultrasound and lab results alone are not considered a visit; they must be linked to an office visit with 
an appropriate practitioner in order to count for this measure. 

The organization must use one date (date of delivery or EDD) to define the start and end of the first trimester. If 
multiple EDDs are documented, the organization must define a method to determine which EDD to use, and use 
that date consistently. The LMP may not be used to determine the first trimester. 

A Pap test alone does not count as a prenatal care visit for the administrative and hybrid specification of the 
Timeliness of Prenatal Care rate, but is acceptable for the Postpartum Care rate. A colposcopy alone is not numerator 
compliant for either rate. 

The intent is that a visit is with a PCP or OB/GYN. Ancillary services (lab, ultrasound) may be delivered by an ancillary 
provider. 

The intent is to assess whether prenatal and preventive care was rendered on a routine, outpatient basis rather than 
assessing treatment for emergent events. 

Refer to Appendix 3 for the definition of PCP and OB/GYN and other prenatal practitioners. 


31 


MO HealthNet Managed Care External Quality Review Section 3 
Supplemental Report — 2016 Performance Measures 





Data Elements for Reporting 
Organizations that submit HEDIS data to NCQA must provide the following data elements. 


Table PPC-1I/2: Data Elements for Prenatal and Postpartum Care 


| Administrative Hybrid 


rates rates 
rates rates 
the 2 rates rates 
e Number of numerator events by administrative data in Lae 
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i ~dlaniaiees ial CN ch 
size rates 
ii I ll 
rates 
dec ha 
rates 
e Number of numerator events by administrative data in For each of the 2 
Oa alae) 
ee ee ey 
rates 
ee Se 
valid data errors rates 
decane ea 
excluded rates 
ar ceca 
rates 
a ee ee 
rates 
the 2 rates rates 
aaa 
rates 
the 2 rates rates 
rates rates 
rates rates 





32 


MO HealthNet Managed Care External Quality Review Section 3 


Supplemental Report — 2016 Performance Measures 


EMERGENCY DEPARTMENT VISITS (EDV) 


ED Visits (count of visits): Medical Diagnoses 


Use MODIFIED HEDIS Administrative specifications for the “Ambulatory Care (AMB)” measure. 
DO NOT use Hybrid specifications. MODIFY the measure by using MHD-specified age groups. 
Report the count of ED visits for age groups 0-12, 13-17, 18-64, and 65+. 


ED Visits (count of visits): Behavioral Health Diagnoses 


The count of emergency department VISITS for behavioral health reasons during the designated 
time period for health plan members. Use MODIFIED HEDIS specs for MPT - Mental Health 
Utilization as described below. Count emergency department VISITS not PATIENTS or EPISODES 
OF CARE. Do not separate patients by gender. Since the HEDIS specs lump Outpatient and ED 
visits together, modify the specs to separate these for this measure. Replace the “Outpatient and 
ED” part of the “Calculations” section of the HEDIS Mental Health Utilization specs with the 


following: 


ER Services 

Report ED claims/encounters in conjunction with a PRINCIPAL mental health diagnosis. 

Any of the following code combinations meet criteria: 
*ED Value Set WITH Mental Health Diagnosis Value Set. (NOTE: Although HEDIS requires 
this to be billed by a mental health practitioner, we do NOT. Any practitioner is 


acceptable.) 


*MPT Outpatient/ED Value Set AND Mental Health Diagnosis Value Set. HOWEVER: 
MODIFY the MPT Outpatient/ED POS Value Set by including ONLY POS=23. EXCLUDE all 
other POS values.) 


Include services provided by physicians and non-physicians. 


Only include observation stays that do not result in an inpatient stay. 


Report the count of ED visits for age groups 0-12, 13-17, 18-64, and 65+. 
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Except for the above modifications, calculate the measure as written in the HEDIS specifications. 
(This specification is the same as for “ED Utilization (count of members): Behavioral Health 
Diagnoses” below, except that you are counting VISITS and not PATIENTS.) 


ED Visits (count of visits): Substance Use Disorders 


The count of emergency department VISITS for substance abuse reasons during the designated time 
period for health plan members. Use MODIFIED HEDIS specs for IAD - Identification of Alcohol 
and Other Drug Services as described below. Count emergency department VISITS not PATIENTS 
or EPISODES OF CARE. Do not separate patients by gender. Since the HEDIS specs lump 
Outpatient and ED visits together, we need to modify the specs to separate these for this measure. 
Replace the “Outpatient and ED” part of the “Calculations” section of the HEDIS Identification of 


Alcohol and Other Drug Services criteria with the following: 


SA ER Services 
Report ED claims/encounters in conjunction with a PRINCIPAL chemical dependency diagnosis. 
(NOTE: HEDIS asks for ANY chemical dependency diagnosis; we are asking for PRINCIPAL). Any 


of the following code combinations meet criteria: 


*ED Value Set WITH Chemical Dependency Value Set. 
*IAD Outpatient/ED Value Set AND Chemical Dependency Value Set. HOWEVER: 
MODIFY the IAD Outpatient/ED POS Value Set by including ONLY POS=23. EXCLUDE all 
other POS values.) 

Include services provided by physicians and non-physicians. 

Only include observation stays that do not result in an inpatient stay. 


Report the count of ED visits for age groups 0-12, 13-17, 18-64, and 65+. 
Except for the above modifications, calculate the measure as written in the HEDIS specs. (This spec 


is the same as for “ED Utilization (count of members): Substance Use Disorders” below, except that 


you are counting VISITS and not PATIENTS.) 
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EMERGENCY DEPARTMENT UTILIZATION (EDU) 


ED Utilization (count of members): Medical Diagnoses 

The count of health plan MEMBERS accessing emergency department services for medical reasons. 
Use MODIFIED HEDIS Administrative specifications for the “Ambulatory Care (AMB)” measure. 
DO NOT use Hybrid specifications. MODIFY the measure by reporting the unique count of 
MEMBERS accessing ED services, rather than the total count of ED VISITS. 


Report the unique count of MEMBERS accessing ED services for age groups 0-12, 13-17, 18-64, and 
65+. 


ED Utilization (count of members): Behavioral Health Diagnoses 

The count of health plan members accessing emergency department services for behavioral health 
reasons. Use MODIFIED HEDIS specs for MPT - Mental Health Utilization as described below. Do 
not separate patients by gender. Since the HEDIS specs lump Outpatient and ED visits together, 
modify the specs to separate these for this measure. Replace the “Outpatient and ED” part of the 


“Calculations” section of the HEDIS Mental Health Utilization specs with the following: 


ER Services 

Report ED claims/encounters in conjunction with a PRINCIPAL mental health diagnosis. Any of the 

following code combinations meet criteria: 
*ED Value Set WITH Mental Health Diagnosis Value Set. (NOTE: Although HEDIS requires 
this to be billed by a mental health practitioner, we do NOT. Any practitioner is 
acceptable.) 
*MPT Outpatient/ED Value Set AND Mental Health Diagnosis Value Set. HOWEVER: 
MODIFY the MPT Outpatient/ED POS Value Set by including ONLY POS=23. EXCLUDE all 
other POS values.) 


Include services provided by physicians and non-physicians. 

Only include observation stays that do not result in an inpatient stay. 

Report the unique count of MEMBERS accessing ED services for age groups 0-12, 13-17, 18-64, and 
65+. 


Except for the above modifications, calculate the measure as written in the HEDIS specs. 
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ED Utilization (count of members): Substance Use Disorders 


The count of health plan members accessing emergency department services for substance abuse 
reasons. Use MODIFIED HEDIS specs for IAD - Identification of Alcohol and Other Drug Services 
as described below. Do not separate patients by gender. Since the HEDIS specs lump Outpatient 
and ED visits together, separate for this measure. Replace the “Outpatient and ED” part of the 
“Calculations” section of the HEDIS Identification of Alcohol and Other Drug Services criteria with 


the following: 


SA ER Services 
Report ED claims/encounters in conjunction with a PRINCIPAL chemical dependency diagnosis. 
(NOTE: HEDIS asks for ANY chemical dependency diagnosis; we are asking for PRINCIPAL). Any 


of the following code combinations meet criteria: 


*ED Value Set WITH Chemical Dependency Value Set. 

*IAD Outpatient/ED Value Set AND Chemical Dependency Value Set. HOWEVER: 
MODIFY the IAD Outpatient/ED POS Value Set by including ONLY POS=23. EXCLUDE all 
other POS values.) 


Include services provided by physicians and non-physicians. 
Only include observation stays that do not result in an inpatient stay. 
Report the unique count of MEMBERS accessing ED services for age groups 0-12, 13-17, 18-64, and 


65+. 


Except for the above modifications, calculate the measure as written in the HEDIS specs. 
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METHODS OF CALCULATING PERFORMANCE MEASURES 

The HEDIS technical specifications allow for two methods of calculating performance measures: |) 
the Administrative Method and 2) the Hybrid Method. Each year one of the measures selected for 
this review, allows for Administrative or Hybrid methods of review. The two remaining measures 


are each calculated using the Administrative Method only. 


The Administrative Method involves examining claims and other databases (administrative data) to 
calculate the number of members in the entire eligible population who received a service (e.g., 
Prenatal and Postpartum Care). The eligible population is defined by the HEDIS technical 
specifications or MO HealthNet defined standards. Those cases in which administrative data show 
that the member received the service(s) examined are considered “hits” or “administrative hits.” 
The HEDIS technical specifications provide acceptable administrative codes for identifying an 


administrative hit. 


For the Hybrid Method, administrative data are examined to select members eligible for the 
measure. From these eligible members, a random sample is taken from the appropriate 
measurement year. Members in the sample are identified who received the service(s) as evidenced 
by a claim submission or through external sources of administrative data (e.g., State Public Health 
Agency Vital Statistics or Immunization Registry databases). Those cases in which an administrative 
hit cannot be determined are identified for further medical record review. Documentation of all or 
some of the services in the medical record alone or in combination with administrative data is 


considered a “hybrid hit.” 


Administrative hits and hybrid hits are then summed to form the numerator of the rate of members 
receiving the service of interest (e.g., appropriate doctor’s visit). The denominator of the rate is 
represented by the eligible population (administrative method) or those sampled from the eligible 
population (hybrid method). A simple formula of dividing the numerator by the denominator 
produces the percentage (also called a “rate”) reported to MO HealthNet and the SPHA. 
Additional guidance is provided in the HEDIS Technical Specifications: Volume 23 for appropriate 
handling of situations involving oversampling, replacement, and treatment of contraindications for 


services. 





3 National Committee for Quality Assurance. HEDIS 2015, Volume 2: Technical Specifications. Washington, 
D.C.: NCQA. 
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TIME FRAME 

The proper time frame for selection of the eligible population for each measure is provided in the 
technical specifications. For the measures selected, the “measurement year” referred to calendar 
year prior to the review year. All events of interest (e.g. follow-up visits) must also have occurred 


during the calendar year prior to the review year. 


PROCEDURES FOR DATA COLLECTION 

The HEDIS technical specifications for each measure validated are reviewed by the EQRO Project 
Director and the EQRO Research Analyst. Extensive training in data management and programming 
for Healthcare quality indices, clinical training, research methods, and statistical analysis expertise 
were well represented among the personnel involved in adapting and implementing the Validating of 
Performance Measures Protocol to conform to the HEDIS, MO HealthNet, and SPHA requirements 
while maintaining consistency with the Validating Performance Measures Protocol. The following 
sections describe the procedures for each activity in the Validating Performance Measures Protocol 


as they were implemented for the HEDIS measures validated. 


Pre-On-Site Activity One: Reviewer Worksheets 
Reviewer Worksheets are developed for the purpose of conducting activities and recording 
observations and comments for follow-up at the site visits. These worksheets are reviewed and 
revised to update each specific item with the current year’s HEDIS technical specifications. Project 
personnel meet regularly to review available source documents and develop the Reviewer 
Worksheets for conducting pre-on-site, on-site, and post-on-site activities as described below. 
These reviews formed the basis for completing the CMS Protocol Attachments (V, VII, X, XII, XIll, 
and XV) of the Validating Performance Measures Protocol for each measure and MCHP. Source 
documents used to develop the methods for review and complete the Attachments included the 
following pertinent to the current review year: 

e HEDIS Data Submission Tool (DST) 

e HEDIS Road Map 

e HEDIS Audit Report 

e HEDIS SPHA Reports 


e MO HealthNet Data submission report 
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Orientation teleconferences with each MO HealthNet MCHP are conducted annually by the EQRO. 
The purpose of this orientation conference is to provide education about the Validating 
Performance Measures protocol and the EQRO’s submission requirements. All written materials, 
letters and instructions used in the orientation are reviewed and approved by MO HealthNet in 
advance. Prior to the teleconference calls, the MCHPs are provided information on the technical 
objectives, methods, procedures, data sources, and contact information for EQRO personnel. The 
MCHPs were requested to have the person(s) responsible for the calculation of that year’s HEDIS 
performance measures to be validated in attendance. Teleconference meetings were led by the 
EQRO Project Director, with key project personnel and a representative from the SMA in 
attendance. Provided via the teleconferences is technical assistance focused on describing the 
Validating Performance Measures Protocol; identification of the three measures selected for 
validation each year; the purpose, activities and objectives of the EQRO; and definitions of the 
information and data needed for the EQRO to validate the performance measures. All MCHP 
questions about the process are answered at this time and identified for further follow-up by the 
EQRO if necessary. In addition to these teleconference calls, presentations and individual 
communications with personnel at MCHPs responsible for performance measure calculation are 


conducted. 


Formal written requests for data and information for the validation of performance measures are 
submitted to the MCHPs by the EQRO recognizing the need to provide adequate time for data and 
medical record collection by each MCHP. This information is returned to the EQRO within a 
specific time frame. The letter sent to all MCHPs and the accompanying instructions for submission 


are detailed in Appendices 3 and 4. 


A separate written request is sent to the MCHPs requesting medical records be submitted to the 
EQRO for a sample of cases. These record requests are then submitted by the providers to the 
EQRO. Detailed letters and instructions are mailed to QI/UM Coordinators and MCHP 
Administrators explaining the type of information, purpose, and format of submissions. EQRO 
personnel are available and respond to electronic mail and telephone inquiries and any requested 


clarifications throughout the evaluation process. 


The following are the data and documents requested from MCHPs for the Validating Performance 


Measures Protocol: 
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e HEDIS Data Submission Tool for all three measures for the MO HealthNet Managed Care 
Population only. 

e Prior year’s HEDIS Audit Report. 

e HEDIS RoadMap for the previous HEDIS year. 

e List of cases for denominator with all appropriate year’s HEDIS data elements specified in 
the measures. 

e List of cases for numerators with all appropriate year’s HEDIS data elements specified in the 
measures, including fields for claims data and all other administrative data used. 

e All worksheets, memos, minutes, documentation, policies and communications within the 
MCHP and with HEDIS auditors regarding the calculation of the selected measures. 

e List of cases for which medical records are reviewed, with all required HEDIS data elements 
specified in the measures. 

e Sample medical record tools used for hybrid methods for the three HEDIS measures for the 
MO HealthNet Managed Care population; and instructions for reviewers. 

e Policies, procedures, data and information used to produce numerators and denominators. 

e Policies, procedures, and data used to implement sampling (if sampling was used). Ata 
minimum, this should include documentation to facilitate evaluation of: 

o Statistical testing of results and any corrections or adjustments made after 
processing. 

o Description of sampling techniques and documentation that assures the reviewer 
that samples used for baseline and repeat measurements of the performance 
measures are chosen using the same sampling frame and methodology. 

o Documentation of calculation for changes in performance from previous periods (if 
comparisons were made), including tests of statistical significance. 

e Policies and procedures for mapping non-standard codes, where applicable. 

e Record and file formats and descriptions for entry, intermediate, and repository files. 

e Electronic transmission procedures documentation. (This will apply if the MCHP sends or 
receives data electronically from vendors performing the HEDIS abstractions, calculations or 
data entry) 

e Descriptive documentation for data entry, transfer, and manipulation programs and 
processes. 

e Samples of data from repository and transaction files to assess accuracy and completeness 


of the transfer process. 
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e Documentation of proper run controls and of staff review of report runs. 

e Documentation of results of statistical tests and any corrections or adjustments to data 
along with justification for such changes. 

e Documentation of sources of any supporting external data or prior years’ data used in 
reporting. 

e Procedures to identify, track, and link member enrollment by product line, product, 
geographic area, age, sex, member months, and member years. 

e Procedures to track individual members through enrollment, disenrollment, and possible re- 
enrollment. 

e Procedures used to link member months to member age. 

e Documentation of “frozen” or archived files from which the samples were drawn, and if 
applicable, documentation of the MCHP’s process to re-draw a sample or obtain necessary 
replacements. 

e Procedures to capture data that may reside outside the MCHP’s data sets (e.g. MOHSAIC). 

e Policies, procedures, and materials that evidence proper training, supervision, and adequate 
tools for medical record abstraction tasks. (May include training material, checks of inter- 
rater reliability, etc.) 

e Appendix V — Information Systems Capabilities Assessment for Managed Care Organizations 


and Prepaid Health Plans 


Pre-On-Site Activity Three: Assess the Integrity of the MCHP's Information System 
The objective of this activity is to assess the integrity of the MCHPs’ ability to link data from 
multiple sources. All relevant documentation submitted by the MCHPs is reviewed by EQRO 
personnel. The review protocols require that an Information Systems Capability Assessment (ISCA) 
be administered every other year. The EQRO follows this process and the MCHPs are informed if 
a full ISCA review will occur when the Orientation Conference Calls occur. The results of this 
review are reflected in the final EQRO. EQRO personnel also review HEDIS RoadMap submitted 


by each MCHP. Detailed notes and follow-up questions are formulated for the site visit reviews. 


On-Site Activity One: Assess Data Integration and Control 

The objective of this activity is to assess the MCHPs’ ability to link data from multiple sources and 
determine whether these processes ensure the accurate calculation of the measures. A series of 
interviews and in-depth reviews are conducted by the EQRO with MCHP personnel (including both 


management and technical staff and 3'¢ party vendors when applicable). These site visit activities 
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examine the development and production procedures of the HEDIS performance measures and the 


reporting processes, databases, software, and vendors used to generate these rates. This includes 
reviewing data processing issues for generating the rates and determining the numerator and 
denominator counts. Other activities involve reviewing database processing systems, software, 
organizational reporting structures, and sampling methods. The following are the activities 
conducted at each MCHP: 
e Review results of run queries (on-site observation, screen-shots, test output) 
e Examination of data fields for numerator & denominator calculation (examine field 
definitions and file content) 
e Review of applications, data formats, flowcharts, edit checks and file layouts 
e Review of source code, software certification reports 
e Review HEDIS repository procedures, software manuals 
e Test for code capture within system for measures (confirm principal & secondary codes, 
presence/absence of non-standard codes) 
e Review of operating reports 
e Review information system policies (data control, disaster recovery) 


e Review vendor associations & contracts 


The following are the type of interview questions developed for the site visits: 
e What are the processes of data integration and control within information systems? 
e What documentation processes are present for collection of data, steps taken and 
procedures to calculate the HEDIS measures? 
e What processes are used to produce denominators? 
e What processes are used to produce numerators? 
e How is sampling done for calculation of rates produced by the hybrid method? 


e How does the MCHP submit the requirement performance reports to the State? 


From the site visit activities, interviews, and document reviews, Attachment V (Data Integration and 


Control Findings) of the CMS Protocol is completed for each MCHP and performance measure 


validated. 
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On-Site Activity Two: Assess Documentation of Data and Processes Used to Calculate 
and Report Performance Measures 


The objectives of this activity are to assess the documentation of data collection, assess the process 
of integrating data into a performance measure set, and examine procedures used to query the data 


set to identify numerators, denominators, generate a sample, and apply proper algorithms. 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment VII (Data and Processes Used to Calculate and Report Performance) of the 
CMS Protocol is completed for each MCHP and measure validated. One limitation of this step is 
the inability of the MCHPs to provide documentation of processes used to calculate and report the 
performance measures due to the use of proprietary software or off-site vendor software and 
claims systems. However, all MCHPs are historically able to provide documentation and flow- 
charts of these systems to illustrate the general methods employed by the software packages to 


calculate these measures. 


On-Site Activity Three: Assess Processes Used to Produce the Denominators 
The objectives of this activity are to: |) determine the extent to which all eligible members are 
included; 2) evaluate programming logic and source codes relevant to each measure; and 3) evaluate 


eligibility, enrollment, age, codes, and specifications related to each performance measure. 


The content and quality of the data files submitted are reviewed to facilitate the evaluation of 
compliance with the HEDIS 2016 technical specifications. The MCHPs consistently submit the 
requested level of data (e.g., all elements required by the measures or information on hybrid or 
administrative data). In order to produce meaningful results, the EQRO requires that all the 


MCHPs submit data in the format requested 


From the site visit activities, interviews, review of numerator and denominator files and document 
reviews, Attachment X (Denominator Validation Findings) of the CMS Protocol is completed for 


each MCHP and the performance measures being validated. 


On-Site Activity Four: Assess Processes Used to Produce the Numerators 
The objectives of this activity are to: |) evaluate the MCHPs’ ability to accurately identify medical 
events (e.g., appropriate doctor’s visits); 2) evaluate the MCHPs’ ability to identify events from other 


sources (e.g., medical records, State Public Immunization Registry); 3) assess the use of codes for 
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medical events; 4) evaluate procedures for non-duplication of event counting; 5) examine time 
parameters; 6) review the use of non-standard codes and maps; 7) identify medical record review 
procedures (Hybrid Method); and 8) review the process of integrating administrative and medical 


record data. 


Validation of the numerator data for all three measures is conducted using the parameters specified 
in the HEDIS Technical Specifications; these parameters applied to dates of service(s), diagnosis 
codes, and procedure codes appropriate to the measure in question. For example, the Annual 
Dental Visit measure requires that all dates of service occurred between January | and December 
3 lof the review year. Visits outside this valid date range were not considered. Similar validation is 
conducted for all three measures reviewed. This numerator validation is conducted on either all 
numerator cases (Administrative Method) or on a sample of cases (Hybrid Method). 

Additional validation for measures being calculated using the Hybrid Method is conducted. The 
Protocol requires the EQRO to sample up to 30 records from the medical records reported by the 
MCHP as meeting the numerator criteria (hybrid hits). In the event that the MCHP reports fewer 
than 30 numerator events from medical records, the EQRO requests all medical records that are 


reported by the MCHP as meeting the numerator criteria. 


Initial requests for documents and data are made on early in the calendar year with submissions due 
approximately six weeks later. The EQRO requires the MCHPs to request medical records from 
the providers. The MCHPs are given a list of medical records to request, a letter from the State 
explaining the purpose of the request, and the information necessary for the providers to send the 
medical records directly to the EQRO. The submission deadline is determined based on the 
original request date, and the date of the final receipt based on that date. The record receipt rate is 


historically excellent. In recent years the EQRO has received 100% of records requested. 


The review of medical records is conducted by experienced RNs currently licensed and practicing in 
the State of Missouri. These RNs participate in the training and medical record review process. 
They are required to have substantive experience conducting medical record reviews for HEDIS 


measures. 


A medical record abstraction tool for the HEDIS measures to be reviewed is developed by the 
EQRO Project Director and revised in consultation with a nurse consultant. The HEDIS technical 


specifications and the Validating Performance Measures Protocol criteria are used to develop the 
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medical record review tools and data analysis plan. A medical record review manual and 
documentation of ongoing reviewer questions and resolutions were developed for the review. A 
half day of training is conducted annually by the EQRO Project Director and staff, using sample 
medical record tools and reviewing all responses with feedback and discussion. The reviewer 
training and training manual covered content areas such as Health Insurance Portability and 
Accountability Act (HIPAA), confidentiality, conflict of interest, review tools, and project 
background. Teleconference meetings between the nurses, coders, and EQRO Project Director are 
conducted as needed to resolve questions and coding discrepancies throughout the duration of the 


medical record review process. 


A data entry format with validation parameters was developed for accurate medical record review 
data entry. The final databases are reviewed for validity, verified, and corrected prior to performing 
analyses. All data analyses are reviewed and analyzed by the EQRO Project Director. CMS 
Protocol Attachments XII (Impact of Medical Record Findings) and XIll (Numerator Validation 


Findings) are completed based on the medical record review of documents and site visit interviews. 


On-Site Activity Five: Assess Sampling Process (Hybrid Method) 
The objective of this activity is to assess the representativeness of the sample of care provided. 
e Review HEDIS RoadMap 
e Review Data Submission Tool (DST) 
e Review numerator and denominator files 
e Conduct medical record review for measures calculated using hybrid methodology 
e Determine the extent to which the record extract files are consistent with the data found in 
the medical records 
e Review of medical record abstraction tools and instructions 


e Conduct on-site interviews, activities, and review of additional documentation 


For those MCHPs that calculating one of the identified HEDIS measures via the hybrid methodology, 
a sample of medical records (up to 30) is conducted to validate the presence of an appropriate well- 


child visit that contributed to the numerator. 


On-Site Activity Six: Assess Submission of Required Performance Measures to State 
The objective of this activity is to assure proper submission of findings to MO HealthNet and SPHA. 


The DST is obtained from the SPHA to determine the submission of the performance measures 
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validated. Conversations with the SPHA representative responsible for compiling the measures for 
all MCHPs in the State occurred with the EQRO Project Director to clarify questions, obtain data, 


and follow-up on MCHP submission status. 


Post- On-Site Activity One: Determine Preliminary Validation Findings for each 
Measure 


Calculation of Bias 

The CMS Validating Performance Measures Protocol specifies the method for calculating bias based 
on medical record review for the Hybrid Method. In addition to examining bias based on the 
medical record review and the Hybrid Method, the EQRO calculates bias related to the 
inappropriate inclusion of cases with administrative data that fall outside the parameters described in 
the HEDIS Technical Specifications. For measures calculated using the Administrative Method, the 
EQRO examines the numerators and denominators for correct date ranges for dates of birth and 
dates of service as well as correct enrollment periods and codes used to identify the medical events. 
This is conducted as described above under on-site activities three and four. The estimated bias in 
the calculation of the HEDIS measures for the Hybrid Method is calculated using the following 
procedures, methods and formulas, consistent with the Validating Performance Measures Protocol. 


Specific analytic procedures are described in the following section. 


Analysis 

Once the medical record review is complete, all administrative data provided by the MCHPs in their 
data file submissions for the HEDIS hybrid measure are combined with the medical record review 
data collected by the EQRO. This allows for calculation of the final rate. In order for each event to 
be met, there must be documented evidence of an appropriate event code as defined in the HEDIS 


Technical Specifications. 


For the calculation of bias based on medical record review for the MCHPs using the Hybrid Method 
for the HEDIS measure selected, several steps are taken. First, the number of hits based on the 
medical record review is reported (Medical Records Validated by EQRO). Second, the Accuracy 
(number of Medical Records able to be validated by EQRO/total number of Medical Records 
requested by the EQRO for audit) and Error Rates (100% - Accuracy Rate) are determined. Third, 
a weight for each Medical Record is calculated (100%/denominator reported by the MCHP) as 
specified by the Protocol. The number of False Positive Records is calculated (Error Rate * 


numerator hits from Medical Records reported by the MCHP). This represents the number of 
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records that are not able to be validated by the EQRO. The Estimated Bias from Medical Records is 


calculated (False Positive Rate * Weight of Each Medical Record). 


To calculate the Total Estimated Bias in the calculation of the performance measures, the 


Administrative Hits Validated by the EQRO (through the previously described file validation 


process) and the Medical Record Hits Validated by the EQRO (as described above) are summed and 


divided by the total Denominator reported by the MCHP on the DST to determine the Rate 
Validated by the EQRO. The difference between the Rate Validated by the EQRO and the Rate 
Reported by the MCHP to MO HealthNet and SPHA is the Total Estimated Bias. A positive number 


reflects an overestimation of the rate by the MCHP, while a negative number reflects an 


underestimation. 


Once the EQRO concludes its on-site activities, the validation activity findings for each performance 


measure are aggregated. This involves the review and analysis of findings and Attachments produced 


for each performance measure selected for validation and for the MCHP’s Information System as a 


result of pre-on-site and on-site activities. The EQRO Project Director reviews and finalizes all 


ratings and completed the Final Performance Measure Validation Worksheets for all measures 


validated for each of the MCHPs. Ratings for each of the Worksheet items (0 = Not Met; | = 


Partially Met; 2 = Met) are summed for each worksheet and divided by the number of applicable 


items to form a rate for comparison to other MCHPs. The worksheets for each measure are 


examined by the EQRO Project Director to complete the Final Audit Rating. 


Below is a summary of the final audit rating definitions specified in the Protocol. Any measures not 


reported are considered “Not Valid.” A Total Estimated Bias outside the 95% upper or lower 


confidence limits of the measures as reported by the MCHP on the DST is considered “Not Valid”. 


Fully Compliant: 


Substantially 
Compliant: 


Not Valid: 


Measure was fully compliant with State specifications. 


Measure was substantially compliant with State specifications and 
had only minor deviations that did not significantly bias the 
reported rate. 


Measure deviated from State specifications such that the reported 
rate was significantly biased. This designation is also assigned to 
measures for which the data provided to the EQRO could not be 
independently validated. 


‘Significantly Biased’ was defined by the EQRO as being outside 
the 95% confidence interval of the rate reported by the MCHP on 
the HEDIS 2007 Data Submission Tool. 





47 


MO HealthNet Managed Care External Quality Review Section 4 


Supplemental Report — 2016 Compliance with Regulations 


(this page is intentionally left blank) 


48 


MO HealthNet Managed Care External Quality Review Section 4 
Supplemental Report — 2016 Compliance with Regulations 


4.0 Compliance with Regulations 





49 


MO HealthNet Managed Care External Quality Review Section 4 


Supplemental Report — 2016 Compliance with Regulations 


(this page is intentionally left blank) 


50 


MO HealthNet Managed Care External Quality Review Section 4 
Supplemental Report — 2016 Compliance with Regulations 
PLANNING COMPLIANCE MONITORING ACTIVITIES 

Gathering Information on the MO HealthNet MCHP Characteristics 

Currently there are three MCHPs contracted with the State Medicaid Agency (SMA) to provide 
MO HealthNet Managed Care in three Regions of Missouri. The Eastern Region includes St. 
Louis City, St. Louis County, and twelve surrounding counties. The Western Region includes 
Kansas City/Jackson County and twelve surrounding counties. The Central Region includes 
twenty-eight counties in the center of the state. All three MCHPs serve MO HealthNet 
members in all three regions. These three MCHPs are: Missouri Care, Home State Health, and 


Aetna Better Health of Missouri (Aetna Better Health). 


Determining the Length of Visit and Dates 

On-site compliance reviews are conducted in two days at each MCHP, with several reviewers 
conducting interviews and activities concurrently. Document reviews occur prior to the 
complete on-site review at all MCHPs. Document reviews and the Validation of Performance 
Measures interviews are conducted on the first day of the on-site review. Interviews, 
presentations, and additional document reviews are scheduled throughout the second day, 
utilizing all team members for Validating Performance Improvement Projects, and Monitoring 
Medicaid Managed Care Organizations (MCHPs) and Prepaid Inpatient Health Plans (PIHPs). 
Interviews with Case Managers are conducted as part of the Special Study included in these 
reviews. The time frames for on-site reviews are determined by the EQRO and approved by 


MO HealthNet before scheduling each MCHP. 


Establishing an Agenda for the Visit 

An agenda is developed to maximize the use of available time, while ensuring that all relevant 
follow-up issues are addressed. A sample schedule is developed that specifies times for all 
review activities including the entrance conference, document review, Validating Performance 
Improvement Project evaluation, Validating Performance Measures review, conducting the 
interviews for the Compliance Protocol, and the exit conference. A coordinated effort with 
each MCHP occurs to allow for the most effective use of time for the EQRO team and MCHP 
staff. The schedule for the on-site reviews is approved by MO HealthNet in advance and 


forwarded to each MCHP to allow them the opportunity to prepare for the review. 
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Providing Preparation Instructions and Guidance to the MO HealthNet Health 
Plans 

A letter (see Appendix 12) is sent to each MCHP indicating the specific information and 
documents required on-site, and the individuals requested to attend the interview sessions. The 
MCHPs schedule their own staff to ensure that appropriate individuals are available and that all 


requested documentation is present during the on-site review day. 


OBTAINING BACKGROUND INFORMATION FROM THE STATE MEDICAID 
AGENCY 


Interviews and meetings occur with individuals from MO HealthNet to prepare for the on-site 
review, and obtain information relevant to the review prior to the on-site visits. The 
Compliance Review team members request the contract compliance documents prepared 
annually by MO HealthNet. The information on MCHP compliance with the current MO 
HealthNet Managed Care contract is reviewed, along with required annual submission and 
approval information. This documentation is used as a guide for the annual review although final 
compliance with state contract requirements is determined by MO HealthNet. These 
determinations are utilized in assessing compliance with the Federal Regulations. All 
documentation gathered by MO HealthNet is clarified and discussed to ensure that accurate 
interpretation of their findings is reflected in the review comments and findings. MO 
HealthNet’s expectations, requirements, and decisions specific to the MO HealthNet Managed 


Care Program are identified during these discussions. 


DOCUMENT REVIEW 

Documents chosen for review are those that best demonstrate each MCHP’s ability to meet 
federal regulations. Certain documents, such as the Member Handbook, provide evidence of 
communication to members about a broad spectrum of information including enrollee rights and 
the grievance and appeal process. Provider handbooks are reviewed to ensure that consistent 
information is shared regarding enrollee rights and responsibilities. The MO HealthNet 
Managed Care contract compliance worksheets, specific policies that are reviewed annually or 
that are yet to be approved by the SMA, are reviewed to verify the presence or absence of 
evidence that required written policies and procedures exist meeting federal regulations. Other 
information, such as the Annual Quality Improvement Program Evaluation is requested and 
reviewed to provide insight into the MCHP’s compliance with the requirements of the Quality 


Improvement Strategy, which is an essential component of the MO HealthNet Managed Care 
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contract, and is required by the federal regulations. MCHP Quality Improvement Committee 


meeting minutes are reviewed. 


Case Management and Member Services policies and instructions, as well as training curriculum 
are often reviewed to provide insight into the MCHP’s philosophy regarding case management 
activities. In addition, interviews based on questions from MO HealthNet, and specific to each 
MCHP’s Quality Improvement Evaluation, are conducted with direct services staff and 
administrative staff to ensure that local procedures and practices corresponded to the written 
policies submitted for approval. When it is found that specific regulations are “Partially Met,” 
additional documents are requested of each MCHP. In addition, interview questions are 
developed for identified staff to establish that practice directly with members reflects the 
MCHPs’ written policies and procedures. Interviews with Administrative staff occur to address 
the areas for which compliance is not fully established through the pre-site document review 


process, and to clarify responses received from the staff interviews. 


The following documents were reviewed for all MO HealthNet MCHPs: 
e Annual State contract compliance ratings; 
e Results, findings, and follow-up information from the previous External Quality Review; 
and 


e Annual MO HealthNet MCHP Evaluation, submitted each spring. 


CONDUCTING INTERVIEWS 

After discussions with MO HealthNet, the focus of that year’s Compliance Review is 
determined. This often results in in-depth interviews with Member Services and Case 
Management Staff. The goal of these interviews is to validate that practices at the MCHPs, 
particularly those directly affecting members’ access to quality and timely health care, are in 
compliance with approved policies and procedures. The interview questions are developed 
using the guidelines available in the Compliance Protocol, are focused on areas of concern based 
on each MCHP’s Annual Evaluation, or address issues of concern expressed by MO HealthNet. 
Interviews conducted with administrative and management level MCHP staff provides reviewers 
a clearer picture of the degree of compliance achieved through policy implementation. 
Corrective action taken by each MCHP is determined from previous years’ reviews. This 


process reveals a wealth of information about the approach each MCHP is using to become 
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compliant with federal regulations. The current process of a document review, supported by 
interviews with front line and administrative staff, is developed to provide evidence of a system 
that delivers quality and timely services to members, and the degree to which appropriate 
access was available. The interviews provide reviewers with the opportunity to explore issues 
not addressed in the documentation. Additionally, this approach continues to provide follow-up 
from previous EQRO evaluations. A site visit questionnaire for direct services staff, and a 
separate interview tool for Administrators, is developed for each MCHP annually. The 
questions seek concrete examples of activities and responses that validate that these activities 


are compliant with contractual requirements and federal regulations. 


COLLECTING ACCESSORY INFORMATION 

Additional information used in completing the compliance determination included: discussions 
with the EQR reviewers and MO HealthNet MCHP QI/UM staff regarding management 
information systems; Validating Performance Measures; and Validating Performance 
Improvement Projects. The review evaluates information from these sources to validate MCHP 
compliance with the pertinent regulatory provisions within the Compliance Protocol. These 
findings are documented in the EQR final report and are also reflected in rating 


recommendations. 


ANALYZING AND COMPILING FINDINGS 

The review process includes gathering and analyzing information that directly affects each 
MCHP’s contract compliance and how it relates to compliance with the federal regulations. 
Next, interview questions are prepared, based on the need to investigate if practice exists in 
areas where approved policy is not available, and if local policy and procedures are in use when 
approved policy is not complete. The interview responses and additional documentation 
obtained on-site are then analyzed to evaluate how they contributed to each MCHP’s 
compliance. All information gathered is assessed, re-reviewed and translated into 
recommended compliance ratings for each regulatory provision. This information is recorded 
on the MO HealthNet Managed Care scoring form and can be found in the protocol specific 


sections of this section of the report. 
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REPORTING TO THE STATE MEDICAID AGENCY 

During the meetings with MO HealthNet following the on-site review, preliminary findings and 
comparisons to the previous ratings are presented. Discussion occurs with MO HealthNet staff 
to ensure that the most accurate information is available and to confirm that a sound rationale is 
used in rating determinations. Sufficient detail is included in all worksheets to substantiate any 


rating lower than “Met.” The actual ratings are included in the final report. 


COMPLIANCE RATINGS 

All information gathered prior to the compilation of the final report is utilized is compiling the 
final ratings. This includes the most up-to-date results of MCHP submissions to MO HealthNet 
of policy and procedures that meet or exceed contract compliance. This information is then 
compared to the requirements of each federal regulation to ensure that policy and practice are 
in compliance. MO HealthNet has provided ongoing approval to the EQRO to utilize the 
Compliance Rating System developed during the previous reviews. This system is based on a 
three-point scale (“Met,” Partially Met,” “Not Met”) for measuring compliance, as determined by 
the EQR analytic process. The determinations found in the Compliance Ratings considered MO 
HealthNet contract compliance, review findings, MCHP policy, ancillary documentation, and staff 
interview summary responses validate MCHP practices observed on-site. In some instances, 
MO HealthNet Managed Care contract compliance tool rates a contract section as “Met” when 
policies are submitted, even if the policy has not been reviewed and “finally approved.” If the 
MO HealthNet considers the policy submission valid and rates it as “Met,” this rating is used 
unless practice or other information calls this into question. If this conflict occurs, it is 
explained in the final report documentation. The scale allows for credit when a requirement is 


Partially Met. Ratings were defined as follows: 


All documentation listed under a regulatory provision, or one of its 
components was present. MCHP staff was able to provide responses 
to reviewers that were consistent with one another and the available 
documentation. Evidence was found and could be established that 
the MCHP was in full compliance with regulatory provisions. 


Partially Met: There was evidence of compliance with all documentation 
requirements, but staff was unable to consistently articulate 
processes during interviews; or documentation was incomplete or 
inconsistent with practice. 


Not Met: Incomplete documentation was present and staff had little to no 
knowledge of processes or issues addressed by the regulatory 
provision. 





55 


MO HealthNet Managed Care External Quality Review Section 4 


Supplemental Report — 2016 Compliance with Regulations 


(this page intentionally left blank) 


56 


MO HealthNet Managed Care External Quality Review Appendices 
Supplemental Report — 2016 


Appendices 





57 


MO HealthNet Managed Care External Quality Review Appendices 
Supplemental Report — 2016 


(this page intentionally left blank) 


58 


MO HealthNet Managed Care External Quality Review Appendix | 





Supplemental Report — 2016 MCHP Orientation PowerPoint Slides 


Appendix | - MCHP Orientation PowerPoint Slides 


Orientation Agenda 










Introductions 
Orientation to Technical Methods and 
Objectives of Protocols 

Review of Information, Data Requests, and 
Timeframes 

» Performance Measures 

» Performance Improvement Projects 

= Case Management Special Project 

«= Compliance and Site Visits 

Closing Comments, Questions 


Performance Management 
Solutions Group 
2 Deveson ot Betavices Heath Concepts, nc BHC, 






16 External Quality 
Review for the MO 
HealthNet Managed 
Care Program 















Behavioral Health Concepts, Inc. 


Performance Management Solutions Group 
Amy McCurry Schwartz, EQRO Project Director 
Mona Prater, Assistant Project Director 
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Materials Provided 


Group 
2 Demet Remar eae Comcngms we BC, 






Objectives and Technical Methods 


« Validation of Performance Measures 





«= Case Management Special Project 
« Validation of Performance Improvement Projects 
« Health Plan Compliance 
Requests for information and data 
List of BHC contacts for each protocol 
Presentation 





Protocol Activities 


Information and Data Requests 


Contact Persons 
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Validation of 
Performance Measures 





Performance Management 
Solutions Group 
2 Dron of Bintavecest 







Heath Concepts, nc BHC, 


Measure Validation 
» HEDIS 2016 Prenatal and Post-Partum Care 
» Emergency Department Utilization 
Medical, Behavioral Health, and Substance Use 
» Emergency Department Visits 
Medical, Behavioral Health, and Substance Use 








Validation of 
Performance Measures 








Administrative 


Hybrid method 
» Review up to 30 medical records per measure 
sampled randomly 
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Submission Requirements 
for PM Validation 





Performance Management 
Solutions Group 
2 Divan ot Betanrices Heath Concepts, mc BHC, 





For each of the three measures: 
List of cases for denominator with all elements specified in the measures 
“ Use an appropriate delrniter (e.9., @@ for data that may contain commas or quotation marks). 
Data layout for the files wit be jed in the data request, thes data layout must be used to ensure validity 
Listing of fekis names and descrptions of fields (i.¢.,data dictionary) 
List of cases for numerators with all data elements specified in the measures 
Use an appropmate Gelrniter (e.9., @ for data that may contain commes or quotation marks) 
— layout for the files wil be jed in the data request, thes data layout must be used to ensure validity 
ing of fiakis names and Gescrptions of fields (i.¢., data dictionary) 
2016 HEDIS. Audi Report 


Rpactee A AL WERT) Jo16 BHC EQRO Performance Measure Checklist (Method for Calculating HEDIS 
jleasures 





ist of c of cases | for which medical records were reviewed, with all HEDIS 2016 data elements specified in 
BHC il request Ban pl Plans gather up to 30 records per measure, based on a random sample, and 


Sample medical record tools used for hybrid methods for HEDIS 2016 measures and instructions. 


ae It HEDIS suchors regarding the ca hor sg cop = fed La ——— 


Policies, procedures, data and information used ee =nxalinons and denominators 
Policies, procedures, data used to implement sampling 

Policies and procedures for mapping non-standard codes 

Others as needed 


PLEAte NOTE All materials not submitted in the required format will be rejected and will not 









Case Management 

Special Proj 

' Cases will be reviewed in regards to current 
MHD contract requirements 


Portormance Management 
Setutions Group 
amor tenacera Meum Comcngm nc BHC, 













= Assessment 

« Care Plan 

« Discharge 

« Transition of Care (when applicable) 
| Case Review Tool 


= Specific by case type: i.e. Lead, Prenatal, 
Special Health Care Needs... 
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Purpose and 
Objectives 


To assess the completeness of Case 
Management Records. 

To validate the health plans’ compliance 
with MHD contract requirements for Case 
Management. 

To examine the match between Health Plan 
enrollees in Case Management and those 
enrollees known to MHD that meet Case 
Management criteria. 











Medical Record 
Reviews 








HEDIS 

= Medical record samples requested from 
Health Plans for 1 possible hybrid measure (N 
< 30 per measure; 4 weeks) 


Case Management Special Project 


2 es records sam —— requested from Health Plans 
(N => 45; 4 weeks and onsite) 
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Medical Record 
Reviews (Con 


Reviewed and abstracted by experienced and 
RNs and Social Workers 


Standard abstraction tools 










Validation of Performance 
_ Improvement Projects 







Performance Management : 
Solutions Group 
2 Division ot Bebarvices Health Cononpts, nc BHC, 


Two Performance Improvement 





Projects underway in 2016 
“One clinical 


»One non-clinical 
(Statewide PIP -- ADV) 
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Validation of Performance 
Improvement Projects and 






Performance Management > 
Solutions Group 
amen ot tensors eam Concmpm we BAC, 


















PIP Checklist Elements 
Project narratives, baseline measures, methods, interventions, and planned analyses. 
Examples of information are contained in the CMS protocol, Validating Performance 
Improvement Projects [1 
Phase-in/timeframe for each phase of each PIP[1] 

Problem identification 


Hypotheses 

Evaluation Questions 
Description of intervention(s) 
Methods of sampling, 
Planned analyses 


Baseline data source and data 
Cover letter with clarifying information 


Medical records or other original data sources (if applicable, on-site) 
Additional data as needed 


Bevis 5 (ota) VALI of Health and Human Services, Centers for Medicare & Medicaid 
vices (2012) VAL ee ee ee 
External Quality Review Activities: Final Protocol 


110 September 2012 


| _— _ Health Plan 
Compliance 





Full Compliance review year 


-Enrollee Rights 
» Provider Networks/Directories 
= Transition of Care 
» Health Homes 


Fraud and Abuse 
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Performance Management 
Solutions Group 
2 Divison ot Betavices! 





_ Site Visits 


Heat Comcnpts, nc BHC, 





April 2017 
« Aetna Better Health 
June 2017 
» Home State June 20 & 21 
«= MO Care June 26 & 27 
Health Plan Compliance Reviews 





On-site activities 
» Performance Measure Validation 
» Performance Improvement Project Validation 


» Case Management Interviews 









Health Plan to Health Plan Comparisons: 


» Performance Measure audit findings 
and rates 


« Performance Improvement Project 
element compliance 


» Health Plan Compliance 
» Case Management Special Project 
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eo BHC Team and Coordination 


2 Devise of Bebarvices Heal Concepts, nc BHC, 


BHC Contact 





Behavcral Heath Concepts, inc. 
1804 Southwest Bivd. Sute 0 
Jefferson Cty, MO 65109 


Protocol/ Activity Tel 855-385-3776 Health Plan Contact 


Amy McCurry Schwartz 
amecurryg@ibncinfo.com 


Performance Measures 


Asmy McCurry Schwartz 


amecurryg@ibncenfa com 
Mona Prater 
Assistant, Project Orector 


improvement Projects mprater@bhemfo com 


Amy McCurry Schwartz 
Compliance amecurry@bncinfo.com 
Case Management hone Prater 
Special Project mprater@bheinfo.com 


Army McCurry Schwertz 


amecurry@bncanfo. com 
Mona Prater 


Site Visits mpratergbhcinfo.com 


Asry McCurry Schwartz 
Medical Records amecurry@bncinfo.com 
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Appendix 2 — Performance Improvement Project Worksheets 


Plan Name or ID: 
Name of PIP: 
Dates in Study Period: 


| I. ACTIVITY |: ASSESS THE STUDY METHODOLOGY 


Step |: REVIEW THE SELECTED STUDY TOPIC(S) 


1.1 Was the topic selected through data Met 
collection and analysis of comprehensive aspects | __ Partially 


of enrollee needs, care, and services? Met 
__ Not Met 


___ Unable to 
Determine 


Clinical 

___ Prevention of an acute or chronic condition 
____High volume services 

____Care for an acute or chronic condition 
____ High risk conditions 

Non-Clinical 

___ Process of accessing or delivering care 


1.2 Did the Plan’s PIPs, over time, address a —Met 
broad spectrum of key aspects of enrollee care oo 
i ? 
and services? __Not Met 
___Unable to 
Determine 





Project must be clearly focused on identifying and 
correcting deficiencies in care or services, rather than 
on utilization or cost alone. 
1.3 Did the Plan’s PIPs over time, include all 
enrolled populations (i.e., did not exclude certain = al 
enrollees such as those with special health care Met : 
needs)? __Not Met 


___Unable to 
Determine 





Met 


Demographics: __ Age Range Race 





Gender Total __ Met Partially Met Not Met 
Medical Population: Medicaid Only usd UTD 
Commercial 
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Step 2: REVIEW THE STUDY QUESTION(S) 


_ Met 
__ Partially 
. : M 
2.1 Was the study question(s) stated clearly in foe Met 
writing? __Unable to 
Determine 


Met __ Partially Met Not Met 


Include study question(s) as stated in narrative: Total UTD 





Step 3: Review Selected Indicators 


3.1 Did the study use objective, clearly defined, 

measurable indicators? 
—_ Met 
__ Partially 
Met 
__Not Met 
___Unable to 
Determine 





List Indicators: 


3.2 Did the indicators measure changes in health 
status, functional status, or enrollee satisfaction, | Met 


or processes of care with strong associations __Partially 
Met 

__ Not Met 
___ Unable to 
Determine 


with improved outcomes? 


Are long-term outcomes implied or stated: yes 
__no 
_____ Health Status 

Functional Status 

Member Satisfaction 


Provider Satisfaction Totals 


Met __ Partially Met Not Met 
UTD 
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Performance Improvement Project (PIP) Worksheet 


Step 4: REVIEW THE IDENTIFIED STUDY POPULATION 


4.1 Did the Plan clearly define all Medicaid 








enrollees to whom the study question and Met 
indicators are relevant? __ Partially 
Met 
__ Not Met 
___ Unable to 
Determine 
Demographics Age Range Gender 
Race 
Medical Population: Medicaid Only 
____ Commercial 
4.2 If the study included the entire population, __Met 
did its data collection approach capture all Partially 
enrollees to whom the study question applied? Met 
__Not Met 
___Unable to 
Determine 
Methods of identifying participants: Utilization 
data 
Referral . 
Sell identiieabion oe — Partially Met Not Met 
Other 
Step 5: REVIEW SAMPLING METHODS 
5.1 Did the sampling technique consider and _ Met 
specify the true (or estimated) frequency of Partially 
occurrence of the event, the confidence interval Met 
. f h ill b __Not Met 
to be used, and the margin of error that will be Unable to 
acceptable? Determine 
Previous findings from any other source: 
literature review 
baseline assessment of indices other 
5.2 Were valid sampling techniques that __Met 
protected against bias employed? a 
et 
__Not Met 
___Unable to 
Determine 
_ Met 
__ Partially 
5.3 Did the sample contain a sufficient number Met 
of enrollees? __Not Met 
___Unable to 
Determine 
N of enrollees in sampling frame 
N of sample Totals 


N of participants (i.e. — return rate) 
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Step 6: REVIEW DATA COLLECTION PROCEDURES 


_ Met 
__ Partially 
6.1 Did the study design clearly specify the data | Met 
to be collected? __Not Met 
___Unable to 
Determine 
__ Met 
__ Partially 
6.2 Did the study design clearly specify the Met 
sources of data? __Not Met 
___Unable to 
Determine 
Sources of data: Member Claims 
Provider Other: 
6.3 Did the study design specify a systematic Met 
method of collecting valid and reliable data that __ Partially 
represents the entire population to which the a sf 
ar . ? a ot et 
study’s indicators apply? ~ “Unable to 
Determine 
6.4 Did the instruments for data collection 
provide for consistent, accurate data collection __ Met 
over the time periods studied? a 
et 
__Not Met 
___Unable to 
Determine 


Instruments used: Survey 


Medical Record Abstracti 
Tool Other: Por eae ner peta records are accessed for the hybrid evaluation 


will be requested. 


6.5 Did the study design prospectively specify a Met 
data analysis plan? __ Partially 
Met 
__Not Met 
___Unable to 
Determine 


Inclusion of a description of how medical 











__ Met 

__ Partially 
6.6 Were qualified staff and personnel used to Met 
collect the data? __Not Met 
___Unable to 


Determine 


Project Leader Name: Title: Role: 








Other team members: Names/Roles: 











Met Partially Met Not Met 
Total Y — 

oe UTD 
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Step 7; ASSESS IMPROVEMENT STRATEGIES 


7.1 Were reasonable interventions undertaken 
to address causes/barriers identified through 


data analysis and QI processes undertaken? 


Describe Intervention(s): Member: 





Providers: Share Provider: 


Plan: 


Step 8: REVIEW DATA ANALYSIS AND INTERPRETATION OF STUDY RESULTS 





_ Met 

__ Partially Met 
__Not Met 
___Unable to 
Determine 


Met Partially Met Not Met 


Totals UTD 





8.1 Was an analysis of the findings performed 

according to the data analysis plan? __Met 
__Partially Met 
__Not Met 
__ Not 
Applicable 
___Unable to 
Determine 

This Element is “Not Met” if study is complete and 

there is no indication of a data analysis plan (see step 

6.5) 

8.2 Were the PIP results and findings presented Met 

accurately and clearly? _ Partially Met 
__Not Met 
__Not 
Applicable 
___Unable to 
Determine 

Are tables and figures labeled? yes __no 

Are they labeled clearly & accurately? yes 

no 

8.3 Did the analysis identify: initial and repeat Met 

measurements, statistical significance, factors __ Partially Met 

that influence comparability of initial and repeat | —_Not Met 

measurements, and factors that threaten —Not 

' land exe | validity? Applicable 

internal and external validity? “Unableeo 
Determine 
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Indicate the time periods of measurements: 





Indicate statistical analysis used: 





Indicate statistical significance level or confidence 
level if available/known: 


99% 95% Unable to determine 





8.4 Did the analysis of study data include an 
interpretation of the extent to which its PIP was 
successful and any follow-up activities? 


Limitations described: 





Conclusions regarding the success of the 
interpretation: 





Recommendations for follow-up: 


Step 9: ASSESS WHETHER IMPROVEMENT IS “REAL” IMPROVEMENT 


9.1 Was the same methodology as the baseline 





Performance Improvement Project (PIP) Worksheet 


_ Met 
__Partially Met 
__Not Met 

__ Not 
Applicable 
___Unable to 
Determine 


Met Partially Met 
____Not Applicable UTD 


Not Met 





Totals 





measurement, used, when measurement was —Met 
__Partially Met 
repeated? Not Met 
__Not 
Applicable 
__Unable to 
Determine 
Ask: Were the same sources of data used? 
Did the use the same method of data 
collection? 
Were the same participants examined? 
Did they utilize the same measurement tools? 
9.2 Was there any documented, quantitative 
improvement in processes or outcomes of care? _Met 
__ Partially Met 
__Not Met 
__Not 
Applicable 
___Unable to 
Determine 
Was there: Increase Decrease 
Statistical significance yes no 
Clinical significance yes no 
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9.3 Does the reported improvement in 
performance have “face” validity; i.e., does the 
improvement in performance appear to be the 
result of the planned quality improvement 
intervention? 


Degree to which the intervention was the reason for 
change 


____No relevance Small Fair High 





9.4 Is there any statistical evidence that any 
observed performance improvement is true 
improvement? 


Weak Moderate Strong 


Step 10: ASSESS SUSTAINED IMPROVEMENT 


10.1 Was sustained improvement demonstrated 
through repeated measurements over 
comparable time periods? 


Were the initial study findings verified upon 
repeat measurement? 





__ Met 

__ Partially Met 
__Not Met 

__ Not 
Applicable 
___Unable to 
Determine 


_ Met 
__Partially Met 
__Not Met 

__ Not 
Applicable 
__Unable to 
Determine 


Partially Met 
___Not Applicable UTD 





Totals 


__ Met 

__ Partially 
Met 

—___Not Met 
__ Not 
Applicable 
___ Unable to 
Determine 


Met 





Total 


Not Met 


Partially Met Not Met 
___Not Applicable UTD 
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ACTIVITY 3. EVALUATE OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: 
SUMMARY OF AGGREGATE VALIDATION FINDINGS AND SUMMARY. 


Conclusions: 
Recommendations: 


Check one: 


O High confidence in reported Plan PIP results 

(1 Confidence in reported Plan PIP results 

(1 Low confidence in reported Plan PIP results 

O Reported Plan PIP results not credible 

(1 Unable to determine — the PIP is new and has produced no results 
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Appendix 3 -— Performance Improvement Project Request Documents 





® Behavioral Health Concepts, Inc. 


1804 Southwest Bivd., Ste D, Jefferson City, MO 65109 (855) 385-3776 


www.bhcinfo.com 





April 10, 2017 


Re: 2016 External Quality Review of the MO HealthNet Managed Care Program 
Performance Improvement Project Submission Request 


Dear 


This letter represents a request for information for the 2016 External Quality Review of 
MO HealthNet Health Plans, conducted by Behavioral Health Concepts, Inc., (BHC). 
With this correspondence, we are requesting submission of all information pertaining to 
the Performance Improvement Projects (PIP) selected for validation for the 2016 review. 
The topics chosen for (MPHC) include: 


e Improving Oral Health (Statewide PIP) 
e Childhood Immunizations 


The due date for submission of this information is May 2, 2017. Please send all information to 
BHC, 1804 Southwest Blvd, Ste D, Jefferson City, MO 65109. 


The requested information should include relevant source data for the EQR process. If 
submitting printed versions, include printouts or copies of all required information. Submit 
information for each PIP to be validated for your Health Plan. You may mark PIP sections. 
Provide separate and distinct information for each PIP. We have included face sheets indicating 
the selected PIPs for your health plan. It is acceptable to submit this information electronically. 


Specific information about the implementation of the protocols can be found in the 
documents previously forwarded to all Health Plans for the EQRO orientation and in the 
corresponding CMS 2012 Protocols for External Quality Review. We look forward to 
working with you to implement the External Quality Review. 


Sincerely, 


Mona Prater, MPA 
EQRO Assistant Project Director 
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Appendix 4 —Performance Measure Data Request 
Request Date: 2/07/2017 


Mail To: 

External Quality Review Submission 
Behavioral Health Concepts, Inc. 
1804 Southwest Blvd., Suite D 
Jefferson City, MO 65109 


When applicable, submit one for each of the three measures: 
e Emergency Department Utilization (EDU) 
e Emergency Department Visits (EDV) 
e Prenatal and Postpartum Care (PPC) 


Unless otherwise indicated, please send all documents on CD or thumb drive using the “tab 
numbers” as titles for each document. If an item is not applicable or not available, please indicate 
this in a file on the CD/drive that corresponds to that tab. 





Please report Regional data for the EDU and EDV measures and please report both 
Statewide and Regional data for PPC. 








Electronic Data Submission Instructions: 
(The file layouts to be used for each measure are detailed on pages 2-5 of this document.) 


e Make all submissions using compact disk or thumb drive formats (CD). Data files submitted via 
e-mail will not be reviewed. Insure that files on the CD are accessible on a Microsoft Windows 
7 workstation environment prior to submitting. 

e All files or CDs must be password protected. Do not write the password on the CD. Please 
email the password separately to amccurry@bhcinfo.com. Do not include the password 
anywhere on the CD, or in any correspondence sent with the CD. 

e Data file formats all need to be ASCII, and readable in a Microsoft Windows 7 
environment. Please be sure to name data columns with the same variable names that appear in 
the following data layout descriptions. 

Please include the column names as the first row of data in the file. 
All files must be @ delimited with no text qualifiers (i.e. no quotation marks around 
text fields). 

e Please ensure that date fields are in MM-DD-YYYY format and contain either a null value or a 
valid date. 

e For fields such as Enroll_Last where a member is still enrolled (and therefore a date has not yet 
been determined), the entry must be a valid future date (i.e. a value of | 2-/2-2300 would be 
acceptable to indicate current enrollment; a value of 12-12-1700 would not.) 

e Files will be accepted only in the specified layout. Please avoid adding extra columns or 
renaming the columns we have requested*. Files submitted in any other form will be 
rejected and not validated. 
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There should be 3 separate data files submitted for each measure: 
File |. Enrollment Data 
File 2. Denominator and numerator file 
File 3. Sample selection (cases that were selected for medical record review; this file is 
submitted for PPC Hybrid measure only) 








Please contact BHC prior to the submission deadline if you have any questions regarding these 
layouts or the data submission requirements, and we will be happy to assist you. 


All files received prior to/on the Priority Due Date will be reviewed by BHC personnel. 


Any glaring errors in data format, column format, etc will be noted and you will be 
allowed to resubmit a corrected file prior to the Final Due Date. After the Final Due 
Date, no new data files will be accepted. 
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Emergency Department Utilization (EDU) and Emergency 
Department Visits (EDV) 


(Modified HEDIS Ambulatory Care (AMB)) 

(Modified HEDIS Mental Health Utilization (MPT)) 

(Modified HEDIS Identification of Alcohol and Other Drug Services (IAD)) 
(Administrative Only) 


Please provide the data that was used to report to MHD on June 30, 2016. 
BHC will be matching the information contained in that report (Healthcare 
Quality Data Template). 


File |. Enrollment Data 
Please provide all enrollment periods for each eligible Managed Care Member to verify 
continuous enrollment and enrollment gaps. 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers Managed Care Health Plan name 
Emergency Department Utilization/Emergency Department 
MEASURE EDU/EDV Visits 
The Missouri Medicaid recipient identification number (not 
DCN Whole numbers only the MCHPs internal tracking number) 
MEMBR_FIRST Any basic text Managed Care Member First Name 
MEMBR_LAST Any basic text Managed Care Member Last Name 
Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) Managed Care Member date of birth 
Numbers only in a correct date 
ENROLL_FIRST format (ex. mm/dd/yyyy) First date of enrollment 
Numbers only in a correct date 
ENROLL_LAST format (ex. mm/dd/yyyy) Last date of enrollment 
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Emergency Department Utilization (EDU) and Emergency 
Department Visits (EDV) 


File 2. Denominator and Numerator Data 


Field Name 
MCHP 


MEASURE 


DCN 
MEMBR_FIRST 
MEMBR_LAST 


DOB 
ED_SER_DATE 
ED_SER_CODE 
ED_PROC_CODE 
ED_POS_CODE 


INPT_ADMIT_DATE 


MEN_DX 
MPT_ED_CODE 


MPT_ED_POS_CODE 


CHEM_DEP_DX 
IAD_ED_CODE 


IAD_ED_POS_CODE 
CODING_TYPE 


ADMIN_HIT 
EXCLUD 
EXCLUD_REASON 


Acceptable Content 
Any basic text and/or numbers 


EDU/EDV 


Whole numbers only 
Any basic text 


Any basic text 

Numbers only in a correct date 
format (ex. mm/dd/yyyy) 
Numbers only in a correct date 
format (ex. mm/dd/yyyy) 


Any basic text and/or numbers 
Any basic text and/or numbers 
Any basic text and/or numbers 


Numbers only in a correct date 
format (ex. mm/dd/yyyy) 


Any basic text and/or numbers 
Any basic text and/or numbers 


Any basic text and/or numbers 


Any basic text and/or numbers 
Any basic text and/or numbers 


Any basic text and/or numbers 
C, H, or I 


YorN 
YorN 
Any basic text and/or numbers 


Description 


Managed Care Health Plan name 

Emergency Department Utilization/Emergency 
Department Visits 

The Missouri Medicaid recipient identification number 
(not the MCHPs internal tracking number) 


Managed Care Member First Name 
Managed Care Member Last Name 


Managed Care Member date of birth 


Emergency Department Date of service 

Code used to identify numerator event 

ED visit (ED Value Set Code) 

Code used to identify numerator event 

Procedure Code from the ED Procedure Code Value Set 
Code used to identify numerator event 

ED place of service code (ED POS Value Set) 


Date of inpatient admittance 

Code used to identify numerator event 

PRINCIPAL mental health diagnosis (Mental Health 
Diagnosis Value Set) 


Procedure Code from the MPT Outpatient/ED Value Set 
Place of service code (MPT Outpatient/ED POS Value 
Set) 

Code used to identify numerator event 

PRINCIPAL chemical dependency diagnosis (Chemical 
Dependency Value Set) 


Procedure Code from the IAD Outpatient/ED Value Set 
Place of service code (IAD Outpatient/ED POS Value 
Set) 

Type of coding system: C=CPT Codes; H=HCPCS/CDT-3 
Codes*; I=ICD-9-CM (ICD-10) Codes. 

Administrative numerator event (positive case "hit"): 
y=yes; n=no 

Was the case excluded from denominator Y=Yes; N=No 


Reason for exclusion 


* CDT is the equivalent dental version of the CPT physician procedural coding system. 
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Prenatal and Postpartum Care (PPC) 


File 1. Enrollment Data 
Please provide all enrollment periods for each eligible MO HealthNet Member to verify 


continuous enrollment and enrollment gaps. 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers MO HealthNet Managed Care Organization name 
MEASURE PPC Prenatal and Postpartum Care 
The Missouri Medicaid recipient identification number (not 

DCN Whole numbers only the MCOs internal tracking number) 
MEMBR_FIRST Any basic text MO HealthNet Member First Name 
MEMBR_LAST Any basic text MO HealthNet Member Last Name 

Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) MO HealthNet Member date of birth 


ENROLL_FIRST 


ENROLL_LAST 


File 2. Denominator and Numerator Data 


Numbers only in a correct date 
format (ex. mm/dd/yyyy) 
Numbers only in a correct date 
format (ex. mm/dd/yyyy) 


First date of enrollment 


Last date of enrollment 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers MO HealthNet Managed Care Organization name 
MEASURE PPC Prenatal and Post-Partum Care 
The Missouri Medicaid recipient identification number 

DCN Whole numbers only (not the MCOs internal tracking number) 
MEMBR_FIRST Any basic text MO HealthNet Member First Name 
MEMBR_LAST Any basic text MO HealthNet Member Last Name 

Numbers only in a correct date 
DOB format (ex. mm/dd/yyyy) MO HealthNet Member date of birth 

Numbers only in a correct date 
SER_DATE format (ex. mm/dd/yyyy) Date of service ( 
SER_CODE Any basic text and/or numbers Code used to identify numerator event 


CODING_TYPE 





ADMIN_HIT 
EXCLUD 


C, D, L, U, Ior 110 


YorN 


YorN 
EXCLUD_REASON | Any basic text and/or numbers 





Type of coding system: C=CPT Codes; D=DRGs; 
L=LOINC; U=UB-92 Revenue Codes; I=ICD-9-CM 
Codes; I110=ICD-10-CM or ICD-10-PCS codes. 





Administrative numerator event (positive case "hit"): 
y=yes; n=no 


Was the case excluded from denominator Y=Yes; N=No 
Reason for exclusion 
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Prenatal and Postpartum Care (PPC) 


File 3. For Hybrid method ONLY - please provide a listing of the cases 
selected for medical record review. Use the following layout: 


Field Name Acceptable Content Description 
MCHP Any basic text and/or numbers MO HealthNet Managed Care Organization name 
MEASURE PPC Prenatal and Postpartum Care 

The MO HealthNet Medicaid recipient identification number 
DCN Whole numbers only (not the MCOs internal tracking number) 
MEMBR_FIRST Any basic text MO HealthNet Member First Name 
MEMBR_LAST Any basic text MO HealthNet Member Last Name 

Numbers only in a correct date 

DOB format (ex. mm/dd/yyyy) MO HealthNet Member date of birth 

Medical record review status: 
MR_STATUS RorNRorsS R = reviewed; NR = not reviewed; S = substituted 
PROVIDER_NAME | Any basic text and/or numbers Primary Care Provider who supplied the record 
PROVIDER_ID Any basic text and/or numbers Primary Care Provider identification number 
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2016 External Quality Review of the Missouri Managed Care Program 


Performance Measure Validation Submission Requirements 


Instructions: The following listing includes relevant source data for the EQR process. Please submit information on a CD 
or thumb drive. Each file on the CD or thumb drive should correspond to the tab number and description in 
the spreadsheet below. Within each file, include information specific for each of the three measures for the 
Managed Care population. Some items may not apply. For example, if you do not use a HEDIS vendor and 
perform measure calculations on site, then you may not have documentation of electronic record 
transmissions. These items apply to processes, personnel, procedures, databases and documentation 
relevant to how the MCHP complies with HEDIS measure calculation, submission and reporting. 


If you have any questions about this request, contact Amy McCurry Schwartz, EQRO Project Director, 
amy.mcecurry@bhceqro.com 





Check submitted Use this field to indicate whether you have submitted this information. If you are not submitting the particular 
information, please indicate “NA”. You may have submitted the content by other means either on the BAT or as part of 
some other documentation. If so, indicate “submitted”, and reference the document (see below). 

Name of Source Please write the name of the document you are submitting for the item. If you are submitting pages from a procedure 


Document manual, indicate so by writing "HEDIS submission manual, pages xx — xx." 

MCHP Comments Use this space to write out any concerns you may have or any clarification that addresses any issues or concerns you 

may have regarding either the items requested or what you submitted in the response. 

Reviewed By (BHC use) This space will be for BHC staff use. The purpose will be for tracking what is received and what is not received. It will 
not indicate whether the documents actually address the specific issue. 
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Tab | Performance Measures Check if | Name of Source Document MCHP Reviewed 
Submitted Comments by (BHC 
or NA use) 





HEDIS 2016 Data Submission 
Tool (MO DHSS 2016 Table B 
HEDIS Data Submission Tool 
and/or NCQA IDSS Submission 
Tool) for the PPC measure for 
the MO HealthNet Managed 
Care Population only. Do not 
include other measures or 
populations. 





HEDIS 2016 Audit Report. This 
is the HEDIS Performance Audit 
Report for the Managed Care 
Program product line and the 
three measures to be validated 
(complete report). If the HEDIS 
measure to be validated was not 
audited or if it was not audited 
for the Managed Care Program 
population, please send the 
report, as it contains 
Information Systems Capability 
Assessment information that 
can be used as part of the 
Protocol. 











RoadMap for HEDIS 2016. The 
information submitted for the 
RoadMap will include 
descriptions of the process for 
calculating measures for the MO 
HealthNet Managed Care 
Program population. 
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Tab 


Performance Measures 


Check if 
Submitted 
or NA 


Name of Source Document 


MCHP 
Comments 


Reviewed 
by (BHC 
use) 





List of cases for denominator 
with all data elements specified 
in the measures. 





List of cases for numerators 
with all data elements specified 
in the measures, including fields 
for claims data and MOHSAIC, 
or other administrative data 
used. Please note that one of 
the review elements in the 
Protocol is: The “MCO/PIHP 
has retained copies of files or 
databases used for performance 
measure reporting, in the event 
that results need to be 
reproduced.” 











List of cases for which medical 
records were reviewed, with all 
HEDIS 2016 data elements 
specified in the measures. Based 
ona random sample, BHC will 
request MCHPs to gather a 
maximum of 30 records per 
measure and submit copies of 
the records requested to BHC. 
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Tab 


Performance Measures 


Check if 
Submitted 
or NA 


Name of Source Document 


MCHP 
Comments 


Reviewed 
by (BHC 
use) 





Sample medical record tools 
used if hybrid method(s) were 
utilized for the HEDIS 2016 
Prenatal and Postpartum Care 
measure for the Managed Care 
Program population; and 
instructions for reviewers. 





All worksheets, memos, 
minutes, documentation, 
policies and communications 
within the MCHP and with 
HEDIS auditors regarding the 
calculation of the selected 
measures. (please limit this 
to 30 (two-sided) pages in 
this submission — all other 
information can be 
reviewed onsite, as 
required). 











Policies, procedures, data and 
information used to produce 
numerators and denominators. 
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Performance Measure Data Request 








Tab | Performance Measures Check if | Name of Source Document MCHP Reviewed 
Submitted Comments by (BHC 
or NA use) 
10. Policies, procedures, and data e 


used to implement sampling (if 
sampling was used). Ata 
minimum, this should include 
documentation to facilitate 
evaluation of: 

a. Statistical testing of results 
and any corrections 

or adjustments made 
after processing. 

b. Description of sampling 
techniques and 
documentation that 
assures the reviewer that 

samples used for baseline 
and repeat 

measurements of the 
performance measures 
were chosen using the 
same sampling frame 

and methodology. 

c. Documentation of 

calculation for changes in 
performance from 
previous periods (if 
comparisons were made), 
including tests of 
statistical significance. 











Policies and procedures for 
mapping non-standard codes. 
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Performance Measure Data Request 





Tab | Performance Measures Check if | Name of Source Document MCHP Reviewed 
Submitted Comments by (BHC 
or NA use) 





12. | Record and file formats and 
descriptions for entry, 
intermediate, and repository 
files. 
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Electronic transmission 
procedures documentation. 
(This will apply if the Health 

Plan sends or receives data 
electronically from vendors 
performing the HEDIS 
abstractions, calculations or data 
entry.) 
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Performance Measure Data Request 





Tab 


Performance Measures 


Check if 
Submitted 
or NA 


Name of Source Document 


MCHP 
Comments 


Reviewed 
by (BHC 
use) 





Descriptive documentation for 
data entry, transfer, and 
manipulation of programs and 
processes. 





Samples of data from repository 
and transaction files to assess 
accuracy and completeness of 
the transfer process. 


Documentation of proper run 
controls and of staff review of 
report runs. 











Documentation of results of 
statistical tests and any 
corrections or adjustments to 
data along with justification for 
such corrections or 
adjustments. 
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Tab | Performance Measures Check if | Name of Source Document MCHP Reviewed 
Submitted Comments by (BHC 
or NA use) 
18. Documentation of sources of 


any supporting external data or 
prior years’ data used in 
reporting. 





19. Procedures to identify, track, 
and link member enrollment by 
product line, product, 
geographic area, age, sex, 
member months, and member 
years. 





20. Procedures to track individual 

members through enrollment, 

disenrollment, and possible re- 
enrollment. 





21. Procedures used to link 
member months to member 
age. 
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Performance Measure Data Request 





Tab 


Performance Measures 


Check if 
Submitted 
or NA 


Name of Source Document 


MCHP 
Comments 


Reviewed 
by (BHC 
use) 





22, 


Documentation of “frozen” or 
archived files from which the 
samples were drawn, and if 
applicable, documentation of the 
MCHP’s process to re-draw a 
sample or obtain necessary 
replacements. 





23. 


Procedures to capture data that 
may reside outside the 

MCHP’s data sets (e.g. 
MOHSAIC). 





24. 


Policies, procedures, and 
materials that evidence proper 
training, supervision, and 
adequate tools for medical 
record abstraction tasks. (May 
include training material, checks 
of inter-rater reliability, etc.) 
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Performance Measures to be Calculated for Managed Care Members 
METHOD FOR CALCULATING 2016 PERFORMANCE MEASURES 


Please complete this form and return via email to BHC. Please direct any questions to Amy McCurry 
Schwartz. 


Health Plan 
Date Completed 
Contact Person 
Phone 


Fax 


NCQA Accredited for MO HealthNet Product 
(Yes/No) 


Certified HEDIS Software Vendor and Software 
Record Abstraction Vendor 


EDV EDU PPC 


What was the reporting Date for HEDIS 2016 
Measures? 


What was the Audit Designation (Report/No 
Report/Not Applicable)? 


Was the measure publicly Reported (Yes/No)? 
Did denominator include members who 
switched MCHPs (Yes/No)? 

Did denominator include members who 
switched product lines (Yes/No)? 


Did the denominator include 1115 Waiver 
Members (Yes/No)? 


Were proprietary or other codes (HCPC, NDC) 
used? 


Were exclusions calculated (Yes/No)? 


On what date was the sample drawn? 
Were exclusions calculated (Yes/No)? 
How many medical records were requested? 


How many medical records were received? 


How many medical records were substituted 
due to errors in sampling? 


How many medical records were substituted 
due to exclusions being measured? 
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Appendix 5 - Performance Measures Worksheets 


Final Performance Measure Validation Worksheet: HEDIS 2016 


Prenatal and Postpartum Care (PPC) 


The percentage of deliveries of live births between November 6 of the year prior to the measurement year 
and November 5 of the measurement year. For these women, the measure assesses the following facets 


of prenatal and postpartum care. 


Timeliness of Prenatal Care. The percentage of deliveries that received a prenatal care visit as a 
member of the organization in the first trimester or within 42 days of enrollment in the organization. 


Postpartum Care. The percentage of deliveries that had a postpartum visit on or between 21 and 56 


days after delivery. 
Element Specifications 
Documentation 


Appropriate and complete measurement plans and 
programming specifications exist that include data sources, 
i ic, and computer source code. 


Eligible Population 


None specified. 


Rating Comments 





Continuous 43 days prior to delivery 
Enrollment through 56 days after delivery. 





No allowable gap during the continuous 
Gap enrollment period. 


Anchor date Date of delivery. 





Benefit Medical 





Delivered a live birth on or between 
November 6 of the year prior to the 
measurement year and November 5 of the 
measurement year. Include women who 
delivered in any setting. 

Multiple births. Women who had two 
separate deliveries (different dates of 
service) between November 6 of the year 
prior to the measurement year and 
November 5 of the measurement year count 
twice. Women who had multiple live births 
Event/diagnosis | during one pregnancy count once. 


Sampling 





Sampling was unbiased. 








Sample treated all measures independently. 





Sample size and replacement methods met specifications. 














A division of Behavioral Health Concepts, Inc. 


Performance Management Solutions Group 94 


MO HealthNet Managed Care External Quality Review Appendix 5 
Supplemental Report — 2016 Performance Measure Worksheets 


Numerator 
Data sources used to calculate the numerator (e.g., member 
ID, claims files, medical records, provider files, pharmacy 
records, including those for members who received the 
services outside the MCOs network) are complete and 
accurate. 





Calculation of the performance measure adhered to the 
specification for all components of the numerator of the 
performance measure. 


Documentation tools used were adequate. 








Integration of administrative and medical record data was 
adequate. 





The results of the medical record review validation 
substantiate the reported numerator. 


Denominator 
Data sources used to calculate the denominator (e.g., claims 


files, medical records, provider files, pharmacy records) were 
complete and accurate. 








Reporting 

State specifications for reporting performance measures were 
followed. 

Estimate of Bias 
O - 5 percentage points 
What range percentage points 
defines the > 10 - 20 percentage points 
pee of data > 20 - 40 percentage points 
incompleteness : 
for this > 40 percentage points 
measure? Unable to determine 


What is the Underreporting 


direction of the ; 
bias? Overreporting 









































Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 


Substantially Compliant = Measure was substantially compliant with State specifications and 
had only minor deviations that did not significantly bias the reported rate. 

Not Valid = Measure deviated from State specification such that the reported rate was 
significantly biased. This designation is also assigned to measures for which no rate was 
reported, although reporting of the rate was required. 


Not Applicable = No Members qualified 
Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: 
Emergency Department Utilization 


The percentage of enrolled MO HealthNet Managed Care Program Members who had 
at least one emergency department visit during the measurement year. 
Broken down into three categories of visit: 

Medical; Behavioral Health; and Substance Use 


Ble e s e 
Appropriate and complete measurement plans and 


programming specifications exist that include data 
sources, programming logic, and computer source 














code. 
Age determined as of December 31, 
2016. The measure is reported for 
each of the following age 
stratifications and as a combined 
rate: 
* 0-12 year-olds 
* 13-17 year-olds 
* 18-64 year-olds 

Age * 65+ year-olds 

Enrollment No requirement 

Gap No requirement 

Anchor date None 

Benefit Medical 

Event/diagnosis | None 








Data sources used to calculate the numerator (e.g., 
member ID, claims files, medical records, provider 
files, pharmacy records, including those for members 
who received the services outside the MCOs network) 
are complete and accurate. 





Calculation of the performance measure adhered to 
the specification for all components of the numerator 
of the performance measure. 





Documentation tools used were adequate. 





Integration of administrative and medical record data 
was adequate. 





The results of the medical record review validation 
substantiate the reported numerator. 
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Denominator 


Data sources used to calculate the denominator (e.g., 
claims files, medical records, provider files, pharmacy 
records) were complete and accurate. 

Reporting 


State specifications for reporting performance 
measures were followed. 


Estimate of Bias 


0 - 5 percentage points 
> 5 - 10 percentage points 








> 10 - 20 percentage points 
> 20 - 40 percentage points 
> 40 percentage points 











What range 
defines the 
impact of data 
incompleteness 
for this 
measure? Unable to determine 


What is the Underreporting 
direction of the 
bias? Overreporting 

Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 
Substantially Compliant = Measure was substantially compliant with State 
specifications and had only minor deviations that did not significantly bias the 
reported rate. 




















Not Valid = Measure deviated from State specification such that the reported rate 
was significantly biased. This designation is also assigned to measures for which no 
rate was reported, although reporting of the rate was required. 


Not Applicable = No MC+ Members qualified 


Note: 2 = Met; 0 = Not Met 
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Final Performance Measure Validation Worksheet: 
Emergency Department Visits 


The count of emergency department visits during the measurement year. 
Broken down into three categories: 
Medical; Behavioral Health; Substance Use 


Ble eC a e 
Appropriate and complete measurement plans and 


programming specifications exist that include data sources, 
programming logic, and computer source code. 





Age determined as of December 
31, 2016. The measure is 
reported for each of the following 
age Stratifications and as a 
combined rate: 

* 0-12 year-olds 

* 13-17 year-olds 

* 18-64 year-olds 




















Age * 65+ year-olds 
Enrollment No requirement 
Gap No requirement 
Anchor date None 

Benefit Medical 
Event/diagnosis None 


. « e aVo}o =)e) {= 0 Ca = = ated =maAte = = = = 0 


Data sources used to calculate the numerator (e.g., 
member ID, claims files, medical records, provider files, 
pharmacy records, including those for members who 
received the services outside the MCOs network) are 
complete and accurate. 





Calculation of the performance measure adhered to the 
specification for all components of the numerator of the 
performance measure. 








VE e =100 
Data sources used to calculate the denominator (e.g., 


claims files, medical records, provider files, pharmacy 
records) were complete and accurate. 
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Reporting 
State specifications for reporting performance measures 
were followed. 
Estimate of Bias 
0 - 5 percentage points 
> 5 - 10 percentage points 
> 10 - 20 percentage points 




















What range defines the 








impact of data > 20 - 40 percentage points 


incompleteness for this |> 40 percentage points 
measure? Unable to determine 





Underreporting 





What is the direction of 
the bias? Overreporting 

Audit Rating 
Fully Compliant = Measure was fully compliant with State specifications. 
Substantially Compliant = Measure was substantially compliant with State 
specifications and had only minor deviations that did not significantly bias the 
reported rate. 

















Not Valid = Measure deviated from State specification such that the reported rate was 
significantly biased. This designation is also assigned to measures for which no rate 
was reported, although reporting of the rate was required. 


Not Applicable = No MC+ Members qualified 


Note: 2 = Met; 1 = Partially Met; 0 = Not Met 
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Appendix 6 — Performance Measures Medical Record Request Letter 


HC Behavioral Health Concepts, Inc. 


1804 Southwest Bivd., Suite D, Jefferson City, MO 65109_ (855) 385-3776 (toll-free) 
www.bhcegqro.com 


March 31, 2017 


Subject: 2016 External Quality Review Performance Measure Validation Protocol 
Medical Records Request (hybrid methodology only). 


Due Date: May I, 2017 by 3:00pm 





BHC has reviewed your MCHP’s HEDIS 2016 Prenatal and Postpartum Care (PPC) Measure. 


Please find attached a file containing a listing of the cases related to this HEDIS Measure that 
have been selected for medical record review. Behavioral Health Concepts, Inc. (BHC) requests 
copies of all medical records for these sampled cases. Each medical record supplied should 
contain all the information that contributed to the numerator for the given HEDIS 2016 
Measure. Please forward copies of these medical records to BHC at the following address and 
mark the package as confidential. 


Behavioral Health Concepts, Inc. 
Attn: Amy McCurry Schwartz 
1804 Southwest Blvd., Suite D 
Jefferson City, MO 65109 


If you have any questions, please contact BHC’s External Quality Review team at (855) 385-3776 
x103 or via e-mail: amy.mccurry@bhcegro.com 





Thank you, 


Amy McCurry Schwartz 
EQRO Project Director 


Attachment: 
1) File containing a sample of cases for medical record review 


cc: Mr. Paul Stuve, MO HealthNet Division, Missouri Department of Social Services 
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Appendix 7 — Table of Contents for Medical Record Training Manual 


Table of Contents 

Table of Contents 2 

Background of Project 3 

External Quality Review of Medicaid Managed Care 3 
Qualifications of Reviewers 3 
Confidentiality and Privacy 3 

Conflict of Interest 4 

Record Review Protocols 5 

Purpose of Medical Record Reviews 5 
Process of Request of Medical Records 5 
General Medical Record Review Guidelines 5 
Definition of Medical Record 5 

Claim Form or Claim History 6 

Date Specificity 6 

Organization of Medical Records 6 
Childhood Immunization Status Protocol 6 
Background 6 

Time Period Reviewed 7 

Instructions 7 


PPC Abstraction Tool8 


Requests for Medical Records 16 
Sample Medical Records 18 
Sample Claim Forms/Histories 20 
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Patient Name 


Date of Birth: 
Missing = 
99999999 
Provider 
Name 


Name of MCO 
(Check only 
one) 


Abstractor 
Initials 


Date of 
abstraction 


Data entry 
operator 
initials 


Start Time 


Search the medical record for a Prenatal Visit during the calendar year. 





Prenatal and Postpartum Care 



































































































































Last 

First 
mom d d y 

Last 

First 
LJ Aetna Better Health (1) 
L} Home State Health (2) 
(J Missouri Care (3) 
mom d d y 
h h m m 
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Medical Record Abstraction Tools 





Source of 
Documentation: 


Check One 


Documented 
Components 

of Prenatal Care 
Visit: 


(check all that 
apply) 


Evidence that a 
Prenatal Care 
Procedure was 
performed: 


(Check all that 
apply) 


Evidence that a 
diagnosis of 
pregnancy has 
been established: 





L) Medical Record (1) 
) Claim Form (2) 
QO) Both (3) 
L) None (0) 
Basic physical obstetric exam with auscultation for fetal 
heart tone 
UL) Yes (1) 
L) No (0) 


Pelvic exam with obstetric observations 


L) Yes (1) 

L) No (0) 
Measurement of fundus height 
UO) Yes (1) 

QO) No (0) 


A screening test in the form of an obstetric panel 
OI Yes (1) 
L) No (0) 


TORCH antibody panel alone or a rubella antibody 
test/titler with an Rh incompatibility (ABO/Rh) blood typing 


U) Yes (1) 
L) No (0) 
Echography of a pregnant uterus 
U) Yes (1) 
L) No (0) 


Documentation of LMP (last menstrual period) 
UL) Yes (1) 
L) No (0) 


Documentation of EDD (estimated date of delivery 


UL) Yes (1) 
L) No (0) 
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Documentation of a complete obstetrical history 
UL) Yes (1) 
L) No (0) 


Documentation of prenatal risk assessment and 
counseling/education 


L) Yes (1) 
L) No (0) 


Date of Live Birth m m d d y y y y 


Missing = 99999999 
Not Applicable = 
88888888 



































Notes: 





Search the medical record for a Postpartum Visit during the calendar year 


Source of 
Documentation: Medical Record (1) 
Claim Form (2) 
Both (3) 


None (0) 


Check One 


Oooo 


Date of Delivery m m d d y y y 





Missing = 11119999 
Not Applicable = 11118888 


























Date of Delivery + 21 days m m d d y y y 





Missing = 11119999 
Not Applicable = 11118888 





























Date of Delivery + 56 days m m d d y y y 





Missing = 11119999 
Not Applicable = 11118888 


























Date of Postpartum visit m m d d y y y 





Missing = 11119999 
Not Applicable = 11118888 
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Evidence of one of the Documentation of a pelvic exam 
following: QO) Yes (1) 
L) No (0) 


Documentation of an evaluation of weight, 
blood pressure, breasts and abdomen 


QO) Yes (1) 
L) No (0) 


A notation of "postpartum care" 
QO) Yes (1) 

UO) No (o 
Notes: 





End Time 
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Appendix 9 — Agenda for Site Visits 


MO HealthNet 


Managed Care Program 


External Quality 
Review 





Missouri Care 
SITE VISIT AGENDA 


June 26— Afternoon 


TIME ACTIVITIY ATTENDEES LOCATION 


1:00-4:00 | Case Management Document Mona Prater Conference Room 
Review Lisa Heying — Quiet Location 


1:00 - 1:45 Validation of Performance Amy McCurry 
Measures Schwartz 


Health Plan 
Attendees 


1:45 — 4:00 Compliance Document Review Amy McCurry 
(Information will be Schwartz 
requested as needed) 


June 27 — Morning & Afternoon 

30-9: Introduction -- Opening BHC, Inc. — 
Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 
Health Plan 
Attendees 
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9:00 — 11:00 


12:30 — 1:30 


1:30 — 3:00 


3:00 — 3:15 


3:15 — 3:30 





Case Management & Compliance 


Interviews Case Management 
Staff 


Case Management & Compliance 
Review — Interviews with 
Administrative Staff 


Lunch Break 


Validation of Performance 
Improvement Projects 


Exit Conference Preparation 


Exit Conference 


Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan 
Attendees 


Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan 
Attendees 


Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan 
Attendees 


BHC, Inc. Staff 


Amy McCurry 
Schwartz 
Mona Prater 
Lisa Heying 


Health Plan 
Attendees 


Agenda for Site Visits 
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Appendix 10 — Site Visit Information Request Letter 





Behavioral Health Concepts, Inc. 


1804 Southwest Boulevard, Jefferson City, MO 65109 855-385-3776 (toll free) 
www.behavioralhealthconcepts.com 








June 7, 2017 


RE: SITE VISIT AGENDA AND DOCUMENT REVIEW 

Dear: 

We are finalizing plans for the on-site review of each Health Plan. The following 
information is provided to assist in preparations for the on-site review. We would like 
to make this as efficient as possible for you and your staff. The following information or 


persons will be needed at the time of the on-site review at 


Performance Improvement Projects 





Time is scheduled on afternoon to conduct follow-up questions, review 
data submitted, and provide verbal feedback to the Health Plan regarding the planning, 
implementation, and credibility of findings from the Performance Improvement Projects 
(PIPs). Any staff responsible for planning, conducting, and interpreting the findings of 
PIPs should be present during this time. The review will be limited to the projects and 
findings submitted for 2016. Please be prepared to provide and discuss any new data or 
additional information not originally submitted. 


Performance Measure Validation 





As you know, BHC is in the process of validating the following three performance 
measures: 


e Prenatal and Postpartum Visit (PPV) 
e Emergency Department Utilization (EDU) 
e Emergency Department Visits (EDV) 


BHC is following the CMS protocol for validating performance measures. The goals for 
this process are to: 


= Evaluate the accuracy of Medicaid performance measures reported by the Health 
Plan; and 

« Determine the extent to which Medicaid-specific performance measures calculated 
by the Health Plan followed specifications established by the MO HealthNet 
Division. (Including the HEDIS 2016 Technical Specifications) 
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To complete this process, we will review the following documents while on-site: 
Performance Measure Interviews 


In addition to the documentation reviews, interviews will be conducted with the 
person(s) responsible for: 


=" Overseeing the process of identifying eligible members from Health Plan data 
sources for the measures to be validated; 

= Programming the extraction of required elements from the Health Plan data sources 
for the measures to be validated; 

= Integrity checks and processes of verifying the accuracy of data elements for the 
measures to be validated; 

= Overseeing the process of medical record abstraction, training, and data collection 
for the measures to be validated; and 

=" Contractor oversight and management of any of the above activities. 

=" Demonstration of HEDIS software 

=" Demonstration of the process for extracting data from Health Plan databases 

= Possible data runs for identifying numerator and denominator cases 


Compliance & Case Management Project Review 





The final activity to prepare for during the on-site visit will be the compliance and case 
management review. Documentation review and interviews with MO HealthNet Division 
staff have occurred prior to the on-site visit. This will enable BHC to use the time at the 
Health Plan as efficiently as possible. The following information will be needed at the 
time of the on-site review: 


Compliance Documents 


" Member Handbook 

* 2016 Marketing Plan and materials 

" 2016 Quality Improvement Committee minutes 

= Approved Case Management Policy — Include care management, care coordination, 
and complex case management policy. Please include any practice instructions used, 
if these are separate from policy. 


Compliance 

Interviews with health plan compliance staff will be conducted as needed. 

Case Management Interviews 

The attached agenda requests an interview in the morning with case management staff. 
These interviews are focused on staff members who interact directly with members, and 


who provide case management or disease management services. 


We are asking that the case managers listed be available for the interviews. If there are 
new case managers who would like to attend, but whose names are not included, we 
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would be happy to have them involved in the interviews. Interview questions will include 
general questions regarding practices at the Health Plan, as well as some questions 
generated by the cases we read. 


In some circumstances, it may be necessary to conduct these interviews by telephone. 
In these instances, we request that speaker-phone equipment be available in the 
conference room being utilized by the review team. We realize we are asking for many 
case managers. If someone is not available, it will not be necessary to substitute another 
case manager. A listing of the case managers, who will attend, in person or by 
telephone, will be helpful. Please ensure that the requested staff, when available, be in 
their location at the identified interview time. 


Later interviews are scheduled to include administrative staff. It would be helpful to 
include the following staff: 


* Plan Director 

" Medical Director 

"Quality Assurance Director 

=" Case Management Supervisors or Administrators 
= Utilization Management Director 


These interviews, including required telephone interviews can be scheduled in a 
convenient location in your offices. Document reviews are also scheduled and may 
occur in a separate conference room or meeting space. 


The on-site review team will need to order a working lunch during this visit. If lunch 
facilities are not available, please provide the name and telephone number of a service in 
your vicinity that can accommodate ordering lunch. Your assistance will be appreciated. 


The Health Plan staff involved in any of the referenced interviews or activities, or 
anyone identified by the Health Plan, is welcome to attend the introduction and/or the 
exit interview. 


Again, your assistance in organizing the documents, individuals to be interviewed, and 
the day’s activities is appreciated. If you have questions, or need additional information, 
please let me know. 


Sincerely, 


Mona Prater 
Assistant Project Director 


Cc: Amy McCurry Schwartz, Esq., Project Director 
Paul Stuve, MO HealthNet Division 
Sidney Wilde, MO HealthNet Division 
Lisa Heying, RN Consultant 


Attachment: 
On-Site Review Agenda 
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Appendix || — Compliance Review Scoring Form 


2016 BHC MCHP Compliance Review Scoring Form 


This document is used to score the number of items met for each 
regulation by the MCHP. 


1. Review all available documents prior to the site visit. 


2. Follow-up on incomplete items during the site visit. 
3. Use this form and the findings of Interviews and all completed 
protocols to complete the Documentation and Reporting Tool and rate 
the extent to which each regulation is met, partially met, or not met. 
Scores from this form will be used to compare document compliance 
across all MCHPs. 
0 = Not Met: Compliance with federal regulations could not be 
validated. 
1 = Partially Met: MCHP practice or documentation indicating 
compliance was observed, but total compliance could not be 
validated. 
2 = Met: Documentation is complete, and on-site review produced 
evidence that MCHP practice met the standard of compliance with 
federal regulations. 
























































Contract Bnet 
Compliance Federal and 2015 2014 
Tool Regulation Description Comments | Findings | Rating | Rating 
Subpart C: Enrollee Rights and Protections 
2.6.1(a)1-25, Enrollee 
2.2.6(a), Rights: 
1 | 2.6.2(j) 438.100(a) General Rule 
2.6.1(a)1, Enrollee 
2.9, 2.6.2(j), Rights: Basic 
2 | 2.6.2(n 438.10(b Rule 
Alternative 
Language: 
2.15.2(e), Prevalent 
3 | 2.8.2 438.10(c)(3) Languages 
Language 
and format: 
2.8.2, 2.8.3, Interpreter 
4 | 2.6.2(n)(2) 438.10(c)(4,5) | Services 
Information 
Requirements 
2.6.1(a)1, : Alternative 
5 | 2.6.2(n)1 438.10(d)(1)(i) | Formats 
2.6.1(a)1, 
2.6.2(n)2 - Information 
dot point 35, Requirements 
2.6.2(q), 438.10(d)(1)(ii) | : Easily 
6 | 2.8.2, 2.8.3 and (2) Understood 
28'5), Enrollee 
2.6.1(a)2/3, Rights: 
2.6.2(k)1, Information, 
7 | 2.6.2(n), 438.10(f) Free Choice 
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2.6.2(n)(2), 
2.6.2(q) 
Information to 
Enrollees: 
Physician 
Incentive 
8 | 2.6.2(n)(2) 438.10 (g) Plans 
2.4, 2.4.5, 
2.4.5(a)2-4, Liability for 
2.20.1 (all), Payment and 
9 | 3.5.3(f) 438.10(i) Cost Sharing 
Specific 
Enrollee 
Rights: 
2.2.6(a), Provider- 
2.2.6(b), Enrollee 
1 | 2.6.1(a)(3), 438.100(b)(2)(ii | Communicati 
0 | 2.6.2(j), 2.9.1 | i) ons 
Right to 
Services, 
including right 
2.6.2(j), of refusal. 
1 | 2.30.1, 438.100(b)(2)(i | Advance 
1 | 2.30.2, 2.30.3 | v,v) Directives 
2.6.2(j), 
1) 2.4.8, 2.13, Right to 
2 | 2.14 438.100(b)(3 Services 
Compliance 
with Other 
2.2.6, 2.14.3, State 
2.14.8, 2.14.9 Requirements 
Total Enrollee Rights and 
Protections 
























Subpart D: Quality Assessment and Performance Improvement 





Subpart D: Quality Assessment and Performance Improvement: Access 























Standards 
BBall 
2.6.2(j), Availability of 
2.14.3, Services: 
1 | 2.7.1(g), 438.206(b)(1)(i | Provider 
4 | 3.5.3 -v) Network 
Access to 
2.7.1(e), Well Woman 
1 | 2.7.1(f), Care: Direct 
5 | 2.14.8 438.206(b)(2) Access 
1 Second 
6 | 2.13 438.206(b)(3) Opinions 
Out of 
1 | 2.3.2, 2.3.18, Network 
7 | 2.7.1(bb), 438.206(b)(4) Services: 
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2.12.3, Adequate and 
2.12.4, 2.14.5 Timely 
Coverage 
Out of 
Network 
1 Providers: 
8 | 2.4, 2.20.1(d) | 438.206(b)(5) Cost Sharing 
2.3.14(a)2, 
2.14.1, 
1 | 2.14.4(a-f), 438.206(c)(1)(i_ | Timely 
9 | 2.17.1, 3.5.3 | -vi) Access 
Cultural 
2 | 2.2.6(a)1-3, Consideration 
0 | 2.17.1 438.206(c)(2) Ss 
Primary Care 
and 
Coordination 
2 | 2.14.11, of Healthcare 
1 | 2.3.5(e) 438.208(b) Services 
2.6.2(m), Care 
2 | 2.14.11, Coordination: 
2 | 2.5.3(e) 438.208(c)(1) Identification 
2.12.10, 
2.14.2(c), 
2.14.11, 
2.17.5, 
Attachment 3 
- Children 
with Special Care 
2 | Healthcare Coordination: 
3 | Needs 438.208(c)(2 Assessment 
Care 
Coordination: 
Z| 2A, Ze Treatment 
4 | 2.14.11 438.208(c)(3) Plans 
2.3.8, 2.3.7, 
2 | 2.6.1(k)(3), Access to 
5 | 2.14.6, 2.14.7 | 438.208(c)(4) Specialists 
2.2.1(i), 
2.3.7, 2.7.4, 
2.9.2, 2.10.2, 
2.14.1, 
2 | 2.14.2(a-h), Authorization 
6 | 2.14.2(d)1-2 | 438.210(b) of Services 
Notice of 
2 | 2.15.4, Adverse 
7 | 2.14.2(d)6 438.21 0(c) Action 
2.6.2(k)(3), 
2.14.2(d)6, 
2 | 2.15.4(a-c), Timeframe for 
8 | 2.16.3(e) 438.210(d) Decisions 
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Compensatio 
n for 
Utilization 
2 Management 
9 | 2.17.5(b) 438.210(e) Decisions 
Emergency 
and Pos- 
2.4.8, 2.7.1, stabilization 
2.7.1(y), pgs 24/25 
3 | 2.7.3(v), Rev. 
0 | 2.14.2 438.114 Checklist 
Subpart D: Quality Assessment and Performance Improvement: Structure and 
Operation Standards 
General Rules 
for Credentialing 
2.17.2(n), and 
31 | 2.17.5(c), 2.30.2 438.214(a,b) Recredentialing 
Nondiscriminatio 
n and Provider 
438.214(c) and_ | Discrimination 
32 | 2.2.6(b)(c) 438.12 Prohibited 
Excluded 
33 | 2.31.5 438.214(d) Providers 
Other State 
Requirements: 
Provider 
34 | 2.3.9, 2.3.17 438.214(e) Selection 
2.6.2(n)(2), Disenrollment: 
2.6.2(s)(all), 438.226 and Requirements 
35 | 2.6.2(u 438.56(b)(1-3 and Limitations 
Disenrollment 
2.5.1, 2.5.2, 2.5.6, Requested by 
36 | 2.6.1(g), 2.6.2® 438.56(c) Enrollee 
Procedures for 
Disenrollment -- 
Pgs 29/30 Rev. 
37 | 2.6.2(r,s-1,t) 438.56(d) Checklist 
Timeframe for 
Disenrollment 
38 | 2.6.2(u) 438.56(e) Determinations 
Grievance 
39 | 2.15, 2.15.3(a,b) 438.228 Systems 
2.6.1(a)(18), 
2.16.2(c), Subcontractual 
2.31.2(a)8, 2.31.3, Relationships 
40 | 3.5.1, 3.5.2, 3.5.3 | 438.230(a,b) and Delegation 
Subpart D: Quality Assessment and Performance Improvement: Measurement 
and Improvement 
Adoption of There is 
Practice very little in 
438.236(b)(1-4) | Guidelines the contract 
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compliance 
tool 
regarding 
practice 
guidelines. 

Dissemination of 
Practice 

42 | 2.17.2(d) 438.236(c) Guidelines 
Application of 
Practice 


Guidelines -- Pgs 
32/33 of Rev. 





43 | 2.17.2(d,f) 438.236(d) Checklist 
Quality 
Assessment and 
Improvement 
44 | 2.17.1, 2.17.5 438.240(a)(1) Program 





Basic Elements 
438.240(b)(1) of MCO QI and 















































45 | 2.17.5(d) and 438.240(d) | PIPs 
2.17, 2.17.3, 438.240(b)(2)(c) | Performance 
46 | Attachment 6 and 438.204(c) | Measurement 
Basic elements 
of MCO QI and 
PIPs: Monitoring 
47 | 2.17.5(b 438.240(b)(3 Utilization 
Basic elements 
of MCO QI and 
48 | 2.17.5 438.240(b)(4 PIPs 
Attachment 6 - 
State Quality Program Review 
49 | Strategy 438.240(e) by State 
Health 
Information 
50 | 2.25 438.242(a) Systems 
2.25 (all) - 2.25.1, 
2.25.2(a,b), Basic Elements 
51 | 2.25.3, 2.25.4 438.242(b)(1,2) | of HIS 
Basic Elements 
52 | 2.26.1, 2.29.1 438.242(b)(3) of HIS 
Total Quality Improvement and 
Assessment 
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Subpart F: Grievance Systems 
Grievance and 
Appeals: General 
53 | 2.15 438.402(a) Requirements 
Grievance and 
2.15.2, 2.15.5(a), Appeals: Filing 
54 | 2.15.6(a) 438.402(b)(1) Authority 
Grievance and 
55 | 2.15.6(a) 438.402(b)(2) Appeals: Timing 
2.15.2(a), Grievance and 
2.15.5(a), Appeals: 
56 | 2.15.6(a,b) 438.402(b)(3) Procedures 
Notice of Action: 
15.2(e), Language and 
57 | 2.15.4(a),2.6.2(q) | 438.404(a) Format 
Notice of Action: 
58 | 2.15.4(b) 438.404(b) Content 
Notice of Action: 
59 | 2.15.4(c) 438.404(c) Timing 
Handling of 
Grievances and 
2.15.5(b,c,d), Appeals: General 
60 | 2.15.6(h,i,}) 438.406(a) Requirements 
Handling of 
Grievances and 
2.15.6(g) 2.15.6(h) Appeals: Special 
61 | 2.15.6(i) 2.15.6(j) | 4388.406(b) Requirements 
Resolution and 
notification: 
Grievances and 
2.15.5(e), Appeals - Basic 
62 | 2.15.6(k) 438.408(a) rule 
Resolution and 
notification: 
Grievances and 
Appeals - 
2.15.5(e,f), Timeframes and 
63 | 2.15.6(k-l) 438.408(b,c) extensions 
Resolution and 
notification: 
Grievances and 
2.15.5(e), Appeals - Format 
64 | 2.15.6(k,m) 438.408(d)(e) and content 
Resolution and 
notification: 
2.15.2(i), Grievances and 
65 | 2.15.6(m) 438.408(f) Appeals - 
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Expedited 
resolution of 
2.15.6(n,o 438.410 appeals 


Recordkeeping E 
2.15.3 438.416 and reporting 





This protocol was developed using the CMS MCO Compliance protocol worksheet and cross- 
matching the State of Missouri Eastern/Central Region contract and the State supplied 
Compliance Tool. 
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Appendix |2 — Case Record Review Tool 


BHC Behavioral Health Concepts, Inc. 


1804 Southwest Blvd, Ste D, Jefferson City, MO 65109 855-385-3776 


Health Plan: 

Member Name: 

Case Manager Name(s): 
CM Service Type: 
Reviewer: 


Service Content (note any information about the case, making it unusual or leading to questions 
regarding CM content): 


2016 External Quality Review — Case Review Tool 


After initial referral — 





> Member was contacted and Case Management was initiated. Yes__(if yes proceed to question #1). 
No 

> If No, is there evidence that the member was contacted within time frames? Yes ___ No ___. 

> Were required efforts made to contact the member and establish a relationship? Yes =» No __ 

> Did member refuse services? Yes No F 

> Reason given for not providing case management services: 


When a case is opened for services: 


Introduction to Case Management 
1. Is allidentifying information available, including contact information? Y N 


2. Does narrative contain introductory information to members, such as: 


a. Anexplanation of Case Management services. Y N 
b. The member’s right to accept/reject CM services. Y N 
c. Was obtaining member’s permission a problem? YN N/A 
d. Third party disclosure (obtaining permission to speak to another person/family member about 
medical/referral/CM information) circumstances were explained. Y N 
3. Is the reason for CM services provided? Y N 
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Comprehensive Assessment 
4. Does the case record contain a comprehensive assessment? Y N 


5. Was the assessment completed within required time frames? Y N 

The assessment for CM was within 30 days of enrollment for a new member; 

The assessment for CM was within 30 days of diagnosis for existing members; 

The assessment within 30 days from the date when a member receives the projected discharge date from the 

hospital or rehabilitation facility Y N 
Did the assessment for CM occur within 5 days of admission to a psychiatric hospital or residential substance 
abuse program? Y N 


6. Were additional assessment tools included in the record updating information, particularly if the case was 
opened for an extended period of time (over 12 months)? Y N NA 


Comprehensive Care Planning 





7. Does this record contain care plans? Y N 
a. Did the care plan use clinical practice guidelines? Y__—=—SsN 
b. Is there evidence of member participation in care plan development? Y___siN 
c. Is there evidence that the care plan was discussed, coordinated and/or sent to the member’s PCP? 
Y N 
d. Were care/case plans updated when member’s needs changed or goals achieved? Y N__ 


Type of Service Required 
8. Was the member part of a special program population (SHCNs)? Y N 





a. Did the Case Manager follow Health Plan protocols in serving this member? Y N 
9. Is this member pregnant? Y N 
a. If yes, was case management offered within 15 days of confirmation of pregnancy? Y N 
b. Wasa risk assessment completed? Y N 
c. ls itincluded in the case record? Y N 
10. Is this a lead involved case? Y N 
a. If yes, were case management services initiated within required time frames? Y N 
b. Did the initiation of services indicate which of the following categories the member is in? Y N 


i. 10to 19 ug/dL within 1-3 days 
ii. 20 to 44 ug/dL within 1-2 days 
iii. 45 to 60 ug/dL within 24 hours 
iv. 70Oug/dL or greater — immediately 
c. Did services include follow-up services, as required? Y__—sN 


11. Did the record indicate a diagnosis of: (check any that apply)? 


Cancer 
Cardiac disease 
Chronic pain 
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HepatitisC 

HIV/AIDS 

Sickle Cell Anemia 

Anxiety Disorders 

Pervasive Developmental Disorder 


Members with Special Healthcare Needs without services 
(These may include, but not be limited to private duty nursing, home health, durable medical 
equipment/supplies, and/or a need for hospitalization or institutionalization.) 


The following groups/individuals are at high risk of having a SHCN: 


Individuals with Autism Spectrum disorder 

Individuals eligible for SSI 

Individuals in foster care or other out-of-home placement 

Individuals receiving foster care/adoption subsidy 

Individuals receiving services through a family-centered community-based coordinated care system 
receiving funds under Section 501(a)(1)(D) of Title5 __ 


VVVVV 


Otherdiagnosis:— 


Appropriate Provider and Service Referrals 
12. Were appropriate referrals made for necessary services that were not in place at the time of the assessment, 
or when recommended by the members’ physician/healthcare team? YN N/A 
a. Were individual and/or family support services provided as needed? Y_ oN NA __ 
b. If “Yes” briefly describe: 





13. Were appropriate referrals made for community-based services? Y N N/A 
a. Transportation services? Y _N N/A 


Face to Face Contacts 
14. Is there evidence in the case record that face-to-face contacts occurred, as required? 


Y N N/A 


a. Were face-to-face contacts linked to the goals/outcomes identified in the care plan? Y_ oN =NA__ 





15. Who conducted face-to-face contacts? 
a. Were the outcomes of face-to-face contacts included inthe case record? Y oN _ 


Progress Notes and Required Contacts 
16. Does this case record include progress notes as required? Y N 


17. Is there evidence that at least three (3) substantial contacts were made, directly with the member or their 
representative, prior to case closure? Y N 
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a. Was acommunication plan established with the family/member? Y__N 


PCP Involvement 
18. Do the case notes indicate if the PCP was informed that a case manager was working with the member? 





YN 
a. Were periodic updates given to the PCP when a member’s situation/health changed during CM? 
YN NA 
b. Was the PCP informed when the case management record was closed? Y N Not 
Closed Ss 


19. Was any history or additional information provided to or obtained from the PCP or members of the staff? 
Y N_ 


Case/Care Coordination 
20. Is there any evidence that the member was referred to Disease Management, if appropriate? 
Y N N/A 


21. Is there evidence of care coordination in complex cases, asrequired? Y__————sN NA 

22. Are behavioral health services discussed withthe member? Y__===N NA 

23. When behavioral health services are deemed necessary is the PCP informed? Y__=N NA 

24. Is there evidence of care coordination with the behavioral health CM or provider? YN NA 
Transition Plan and Case Closure 


25. If case closure has occurred, is there evidence that the member has achieved all stated care plan goals and is 
there stabilization of member’s condition, successful links to community support and education, and improved 








member health? Y N N/A 
a) Did the member request to withdraw from either case management or the health plan? Y N 
b) Did lack of contact or compliance occur? Y N 


a. If “Yes” briefly describe: 


c) In this situation was written documentation included indicating plan of attempts to locate/engage 
member? Y N 
a. Examples include: making phone calls before during and after regular working hours; visiting 
the family’s home; sending letters with an address correction request; contact with the PCP, 
WIC office, and other providers or program. 





26. Is there evidence that an appropriate transition of care was offered to the member, and followed at the time a 
case was closed? Y N N/A 
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27. Do proper case closing criteria exist based on the type of case management received? 
Y N N/A 


Additional Questions regarding this case or member situation that should be included in CM interviews: 
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